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Abstract 

We study the power versus distortion trade-off for the distributed transmis- 
sion of a memoryless bi-variate Gaussian source over a two-to-one average-power 
limited Gaussian multiple-access channel. In this problem, each of two separate 
transmitters observes a different component of a memoryless bi-variate Gaussian 
source. The two transmitters then describe their source component to a common 
receiver via an average-power constrained Gaussian multiple-access channel. From 
the output of the multiple-access channel, the receiver wishes to reconstruct each 
source component with the least possible expected squared-error distortion. Our 
interest is in characterizing the distortion pairs that are simultaneously achievable 
on the two source components. 

We present sufficient conditions and necessary conditions for the achievability 
of a distortion pair. These conditions are expressed as a function of the channel 
signal-to-noise ratio (SNR) and of the source correlation. In several cases the 
necessary conditions and sufficient conditions are shown to agree. In particular, 
we show that if the channel SNR is below a certain threshold, then an uncoded 
transmission scheme is optimal. We also derive the precise high-SNR asymptotics 
of an optimal scheme. 



1 Introduction 

We study the power versus distortion trade-off for the distributed transmission of a 
memoryless bi-variate Gaussian source over a two-to-one average-power limited Gaus- 
sian multiple-access channel. In this problem, each of two separate transmitters ob- 
serves a different component of a memoryless bi-variate Gaussian source. The two 
transmitters then describe their source component to a common receiver via an average- 
power constrained Gaussian multiple-access channel. From the output of the multiple- 
access channel, the receiver wishes to reconstruct each source component with the least 
possible expected squared-error distortion. Our interest is in characterizing the distor- 
tion pairs that are simultaneously achievable on the two source components. 

We present sufficient conditions and necessary conditions for the achievability of a 
distortion pair. These conditions are expressed as a function of the channel signal-to- 
noise ratio (SNR) and of the source correlation. In several cases the necessary conditions 
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and sufficient conditions are shown to agree, thus yielding a full characterization of the 
achievable distortions. In particular, we show that if the channel SNR is below a 
certain threshold (that we compute), then an uncoded transmission scheme is optimal. 
We also derive the precise high-SNR asymptotics of an optimal scheme. The uncoded 
result is reminiscent of Goblick's result [lj that for the transmission of a Gaussian source 
over an AWGN channel the minimal squared-error distortion is achieved by uncoded 
transmission. But in our setting uncoded transmission is only optimal for some SNRs. 

Our problem can be viewed as a lossy Gaussian version of the problem addressed 
by Cover, El Gamal and Salehi [2] (see also OH]) in which a bi-variate finite-alphabet 
source is to be transmitted losslessly over a two-to-one multiple-access channel. Our 
problem is also related to the quadratic Gaussian two-terminal source-coding problem 
[5j [6] and to the quadratic Gaussian CEO problem [7J [8]. In both of these problems, 
correlated Gaussians are described distributedly to a central receiver. But, in the 
quadratic Gaussian CEO problem the interest is in reconstructing a single Gaussian 
random variable that underlies the observations of the different transmitters, rather 
than reconstructing each transmitter's observation itself. But more importantly, the 
above two problems are source-coding problems whereas ours is one of combined source- 
channel coding. We emphasize that, as our results show, source-channel separation is 
suboptimal for our setting. 

The problem of transmitting correlated sources over multiple-access channels has 
so far only been studied sparsely. One of the first results is due to Cover, El Gamal and 
Salehi [2] who presented sufficient conditions for the lossless transmission of a finite- 
alphabet bi-variate source over a multiple-access channel. Later, several variations of 
this problem were considered. Salehi [9] studied a lossy versions of the problem with 
a finite-alphabet source and arbitrary distortion measures on each source component. 
For this problem he derived sufficient conditions for the achievability of a distortion 
pair. More recently, another variation where the two source components are binary 
with Hamming distortion and where the multiple-access channel is Gaussian was con- 
sidered by Murugan, Gopala and El Gamal [lOj who derived sufficient conditions for the 
achievability of a distortion pair. Gastpar [11] considered a combined source-channel 
coding analog of the quadratic Gaussian CEO problem. In this problem, distributed 
transmitters observe independently corrupted versions of the same univariate Gaussian 
source. These transmitters are connected to a central receiver by means of a many-to- 
one Gaussian multiple-access channel. The central receiver wishes to reconstruct the 
original univariate source as accurately as possible. For this problem, Gastpar showed 
that the minimal expected squared-error distortion is achieved by an uncoded transmis- 
sion scheme. The extension of our problem to the case where perfect causal feedback 
from the receiver to each transmitter is available is studied in [12] (see also |13j). 

2 Problem Statement 
2.1 Setup 

Our setup is illustrated in Figure [TJ A memoryless bi-variate Gaussian source is con- 
nected to a two-to-one Gaussian multiple-access channel. Each transmitter observes 
one of the source components and wishes to describe it to the common receiver. The 
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Figure 1: Bi-variate Gaussian source with one-to-two Gaussian multiple-access channel. 



source symbols produced at time k £ Z are denoted by (Si k, 5 , 2,fe)- The source output 
pairs {(Si : k, S2 : k)} are independent identically distributed (IID) zero-mean Gaussians 
of covariance matrix 



paia 2 



(1) 



\ P&1&2 &2 J 

where p € [—1,1] and where < of < oo, i € {1,2}. The sequence {Si,fc} of the 
first source component is observed by Transmitter 1 and the sequence {S^fc} of the 
second source component is observed by Transmitter 2. The two source components 
are to be described over the multiple-access channel to the common receiver by means 
of the channel input sequences {^i,fc} and {X2 J k}, where € R and X2,fc G R. The 
corresponding time- A; channel output is given by 



Yk — Xn. + X 2 b + Z k , 



(2) 



where Z k is the time-A; additive noise term, and where {Z k } are IID zero-mean variance- 
N Gaussian random variables that are independent of the source sequence. 

For the transmission of the source {Si^, S^,*;}, we consider block encoding schemes 
and denote the block-length by n and the corresponding n-sequences in boldface, 
e.g. Si = (S^i, 5i ; 2, • • • , <Si,n)- Transmitter i is modeled as a function : R n — > R n 
which produces the channel input sequence Xj based on the observed source sequence 
S? = (<Si,i, Si : 2, • • • , Si : n), i.e. 



x, = ft ] (s 



* e {1,2}. 



(3) 



The channel input sequences are subjected to expected average power constraints 

1 n 

-J2 E i X h]<Pi i€{l,2}, (4) 



k=i 



for some given Pj > 0. 

The decoder consists of two functions <p^ : R ra — > R n , i £ {1,2}, which perform 
estimates Sj of the respective source sequences Sj, based on the observed channel output 
sequence Y, i.e. 



# } (Y) * G{1,2}. 



(5) 



Our interest is in the pairs of expected squared-error distortions that can be achieved 
simultaneously on the source-pair as the blocklength n tends to infinity. In view of this, 
we next define the notion of achievability. 
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2.2 Achievability of Distortion Pairs 



Definition 2.1. Given 01, 02 > 0, p € [—1, 1], Pi, P2 > 0, and iV > we say that the 
tuple {D\,D2,o\,a\, p, Pi , P2 , N) is achievable if there exists a sequence of encoding 
functions {/} , /| } as in ([3]), satisfying the average power constraints and a 

sequence of reconstruction pairs {(p^, <f>2^} as in ([5]), such that the average distortions 
resulting from these encoding and reconstruction functions fulfill 



I " 

lim — > E 

n— >oo 71 ^— ' 

fc=l 



5; 



5, 



< A, 



1,2, 



whenever 

Y = / 1 (n) (S 1 ) + /i n) (S 2 ) + Z, 

and where {(S\k, 5*2, fc)} are IID zero-mean bi-variate Gaussian vectors of covariance 
matrix K55 as in ([I]) and are IID zero- mean variance- iV Gaussians that are inde- 
pendent of {(<Si,fc, S 2>k )}. 

The problem we address here is, for given a\, o~ 2 , p, P\, P2, and N, to find the set 
of pairs (Di,D 2 ) such that (D\, D 2 , o~\, a\, p, Pi, P2, N) is achievable. Sometimes, we 
will refer to the set of all (D\, D2) such that (D\, D2, erf , a 2, p, Pi, Pi, N) is achievable 
as the distortion region associated to (af, o\, p, P\, P 2 , N). In that sense, we will often 
say, with respect to some {a\, 02, p, Pi, P2,A r ), that the pair (Di,£>2) is achievable, 
instead of saying that the tuple {D\, D2, &\, o\, P, Pi, P2, N) is achievable. 



2.3 Normalization 

For the described problem we now show that, without loss in generality, the source law 
given in ([1]) can be restricted to a simpler form. This restriction will ease the statement 
of our results as well as their derivations. 

Reduction 2.1. For the problem stated in Sections 12.11 and 12.21 there is no loss in 
generality in restricting the source law to satisfy 

a\=a\=a' 2 and p€[0,l]. (6) 

Proof. The proof follows by noting that the described problem has certain symmetry 
properties with respect to the source law. We prove the reductions on the source 
variance and on the correlation coefficient separately. 

i) The reduction to correlation coefficients p S [0, 1] holds because the optimal 
distortion region depends on the correlation coefficient only via its absolute 
value \p\. That is, the tuple (Di, D2, (j\ , a\, p, P\,P2,N) is achievable if, and 
only if, the tuple (P^i, D2, a\ , <j\, — p, Pi, P2, N) is achievable. To see this, note 
that if {/1 , /.J , 4>i , 4>2 } achieves the distortion {Di,D 2 ) for the source of 
correlation coefficient p, then {/1 ,f% , 0i ,<j>2 }> wnere 

ff\s 1 ) = A (n) (-S!) and 0S n) (Y) = -^ ] (Y) 
achieves (Di,D2) on the source with correlation coefficient —p. 
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ii) The restriction to source variances satisfying a\ = a\ = a 2 incurs no loss of 
generality because the distortion region scales linearly with the source variances. 
That is, the tuple {D\, D2, <rf, a 2 , p, Pi, P2, N) is achievable if, and only if, for 
every a%, 02 G the tuple (a\Di, (X2D2, u\o\, a.io\, p, P\, P2, N) is achievable. 
This can be seen as follows. If {/-[ , , </>[ ,<j>2 } achieves (Di, D2, o-f, a 2 , p, 
P\,P2, N) , then the combination of the encoders 

fT\Si) = ft\s*/V^), i€{l,2}, 
with the reconstructors 

4> ( r ) (Y) = ^.^ ) (Y), ie{l,2}, 

achieves the tuple (a\Di, OL2D2, ct\a 2 , &2&21 Pi Pii -^2, N). And by an analogous 
argument it follows that if (a±Di, 0,2^2-, ol\o\ , c^c!' Pi ^1 ^2, N) is achievable, 
then also (D\ , D2 , o~\ , a\ , p, Pi , P2 , N) is achievable. □ 

In view of Reduction 12 . 1 1 we assume for the remainder that the source law addition- 
ally satisfies ©. 

2.4 "Symmetric Version" and a Convexity Property 

The "symmetric version" of our problem corresponds to the case where the transmitters 
are subjected to the same power constraint, and where we seek to achieve the same 
distortion on each source component. That is, P\ = P2 = P, and we are interested in 
the minimal distortion 

D*(a 2 ,p,P,N) = inf{£>: (D,D,a 2 ,a 2 , p,P,P,N) is achievable}, 

that is simultaneously achievable on {£1^} and on {S^fc}- I n this case, we define the 
SNR as P/N and seek the distortion D*(o~ 2 , p, P, N), for some fixed a 2 and p, and as 
a function of the SNR. 

We conclude this section with a convexity property of the achievable distortions. 

Remark 2.1. If (Z?i , D 2 , erf , cr|, p, P x , P 2 , N) and (Di ,D2,a 2 ,a^,p,Pi,P 2 , N) are achiev- 
able, then 

(XDi + \Dx, XD 2 + XD 2 , a 2 , alp, XP 1 + APi, AP 2 + AP 2 , n) , 
is also achievable for every A G [0, 1], where A = (1 — A). 

Proof. Follows by a time-sharing argument. □ 

3 Preliminaries: Sending a Bi-Variate Gaussian over an 
AWGN Channel 

In this section we lay the ground for our main results. We study a point-to-point 
analog of the multiple-access problem described in Section 12.11 More concretely, we 
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consider the transmission of a memoryless bi-variate Gaussian source, subject to ex- 
pected squared-error distortion on each source component, over the additive white 
Gaussian noise (AWGN) channel. For this problem, we characterize the power versus 
distortion trade-off and show that below a certain SNR threshold, an uncoded trans- 
mission scheme is optimal. This problem is simpler than our multiple-access problem 
because here source-channel separation is optimal. 



3.1 Problem Statement 

The setup considered in this section is illustrated in Figure El The difference to the 
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Figure 2: Bi-variate Gaussian source with additive white Gaussian noise channel. 

multiple-access problem of Section 12.11 is that now the two source sequences Si and 
S2 are observed and transmitted jointly by one single transmitter rather than by two 
distributed transmitters. Thus, the channel input sequence X is a function f( n ' : M. n x 
W 1 — ► MJ 1 of the source sequences (Si,S2), i.e. 

X = /W(Si,S 2 ). (7) 

This channel input sequence is subjected to an average power constraint 

1 n 

^£EM<P, (8) 

k=l 

for some given P > 0. 

The remainder of the problem statement is as in the multiple-access problem. The 
source law is assumed to be given by (Q]) and to satisfy ©. The reconstruction functions 
are as defined in ([5]), and the achievability of distortion pairs is defined analogously as 
in Section [2. II Our interest is in the set of achievable distortion pairs (Di,!^)- 



3.2 Rate-Distortion Function of a Bi-Variate Gaussian 

Denoting the rate-distortion function of the source {(<Si,jfc, S^fc)} by Rs lt s 2 (Di, D2), the 
set of achievable distortion pairs is given by all pairs {D\,D2) satisfying 

R Sl ,s 2 ( D U D 2) <^log 2 (l + ^j. (9) 
We next compute the rate-distortion function Rs 1 ,s 2 iP\ , D2) ■ 
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Theorem 3.1. The rate- distortion function Rs lt s 2 (Di, D 2 ) is given by 

' ■(■£-) if(D 1 ,D 2 )e9 1 



R Su s 2 (D 1 ,D 2 ) = { 



2 10 &2 I D X D 2 



D 1 D 2 -(p<r 2 -y/(<T 2 -D 1 )(a 2 -D2) 



if(Di,D 2 )e@2 



t I if(D u D 2 )€ 



(10) 



where log^(x) = max{0, log 2 (x)}, D mm = mm{Di, D 2 } and where the regions @\, @ 2 
and 3>3 are given by 



01 = {(D 1 ,D 2 ): 0<Dt< a\l - p 2 ), D 2 >a\l- p 2 ) + p 2 D 1 - 



a 2 (l - p 2 ) <D 1 < a 2 , D 2 > a 2 (l - p 2 ) + p 2 D l , 



D 2 < 



Dl -a 2 (l-p 2 



2> 2 = <(D U D 2 ): 0<Di< a 2 {l - p 2 ), < D 2 < (a 2 (l - p 2 ) - D x ) 



o 2 -D 1 y 



(D U D 2 ) : 0< Di < a 2 (l-p 2 ), 



a 



a z (l - p z ) - D x )— 2 — <D 2 < a 2 (l - p 2 ) + p 2 D i; 



a 2 (l - p 2 ) <D l <a z 



<D 2 <a 2 (l- p 2 )+ p 2 D 1 



Proof. See Appendix [XI 



□ 



The result of Theorem 13.11 was also established independently (and by a different 
proof) in [H] . The regions S>\ , @ 2 , and ^3 are illustrated in Figure El 

Remark 3.1. Let -Rg^-Di) denote the rate-distortion function for the source component 
{Si,k}, i-e., 

« Sl(Dl ) = iiogj(£), 

and let Rs 2 \s 1 (D 2 ) denote the rate-distortion function for {S 2t k} when {Si^} is given 
as side-information to both, the encoder and the decoder, i.e., 

^ 2 |5 1 P 2 ) = ^Og^ f72(1 " / " 



D 2 
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Figure 3: The regions $2, S>3- 

Then, for every (Di,!^) € @2 the rate-distortion function Rs lt s 2 {Dii A2) satisfies 



a) 1 



1 



a 2 (l-p 2 ) 



2 l0g2 UJ + 2 l0g M D 2 
= i2 Sl (Di)+i2s 2 | Sl (£>2), 
where a) holds since for (D±, D2) G ^2 we have -Di, -D2 > 0. 

3.3 Optimal Uncoded Scheme 

As an alternative to the separation-based approach, we now present an uncoded scheme 
that, for all SNR below a certain threshold, is optimal. The optimality of this uncoded 
scheme will be useful for understanding a similar result in the multiple-access problem. 

The uncoded scheme can be described as follows. At every time instant k, the 
transmitter produces a channel input of the form 



a 2 (a 2 + 2pafi + (3 2 ) 



{aS ltk + (3S 2 ,k) fc e {1,2, ... ,n}, 



for some a, (5 £ R. From the resulting channel output Yk, the receiver makes a minimum 
mean squared-error (MMSE) estimate Sf k , i € {1,2}, of the source sample S^k, he., 



5£ fc = E[S iifc |n], iG {1,2}. 
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The corresponding expected distortions on {Si^} and on {52,*:} are 

fiu/ os 2 ^ 2 /3 2 (l ~ P 2 ) + PN(a 2 + 2pq/3 + [3 2 {2 - p 2 )) + jV 2 (q 2 + 2paf3 + /3 2 ) 
° (P + N) 2 (a 2 + 2pa(3 + P 2 ) 

fp, R s _ 2 P 2 « 2 (1 - P 2 ) + PiV(/3 2 + 2pap + a 2 (2 - /> 2 )) + N 2 (a 2 + 2pa/3 + /3 2 ) 
U ^P)- a (P + iV) 2 (a 2 + 2pa/3 + /3 2 ) 

The optimality of this uncoded scheme below a certain SNR-threshold is stated next. 

Proposition 3.1. Let (D\,D2) be an achievable distortion pair for our point-to-point 
setting. If 

^<T(D 1 ,a 2 ,p), (11) 

where the threshold T is given by 

r -^v;- 2 ^;; 1 -/)^? l fv<D 1 <o 2 {i- P 2 ), 

[ +00 else, 

then there exist a* , (3* > such that 

Dl(a*,P*)<Di and D%(a* , (3*) < D 2 . 

Proof. See Appendix [Bj □ 

In the symmetric case, Proposition 13.11 simplifies as follows. 

Corollary 3.1. Let D > be such that (D,D) is an achievable distortion pair for the 
point-to-point problem. If 

1 < (13) 

N ~ \-p y J 

then the pair (D, D) is achieved by the uncoded scheme with time-k channel input 



XUa, a) = J 2(j2 ^ + p) (Si,k + S 2 , k ) for k € {1, 2, ... , n}. 

Corollary 13.11 can also be verified without relying on Proposition 13. 1L This is dis- 
cussed in the following remark. 

Remark 3.2. The distortions resulting from the uncoded scheme with any choice of 
(a, (3) such that a = (3 are 

By evaluating the necessary and sufficient condition of Q for the case where D\ = 
D 2 = D, it follows that this is indeed the minimal achievable distortion for all P/N 
satisfying (fT3"|) . 

This concludes our discussion of the point-to-point problem. 
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4 Main Results 



4.1 Necessary Condition for Achievability of (Di,D 2 ) 

Theorem 4.1. A necessary condition for the achievability of a distortion pair (D\,D2) 
is that 

Rs 1 ,s 2 (D 1 ,D 2 ) < -log a I 1 + — . (14) 

Proof. See Appendix [Cj □ 

Remark 4.1. Theorem 14.11 can be extended to a wider class of sources and distortion 
measures. Indeed, if the source is any memoryless bi-variate source (not necessarily 
zero-mean Gaussian) and if the fidelity measures d\(si, si), d 2 (s 2 , §2) > that are used 
to measure the distortion in reconstructing each of the source components are arbitrary, 
then the pair (Z}i,-D 2 ) is achievable with powers P\,P 2 only if 



mf /(Si,S 2 ;5i,S 2 ) < -log I 1 + r= , (15) 

E[di(Si,Si )]<Di 
E[d2(5 2 ,5 2 )]<D2 

where p max is the Hirschfeld-Gebelein-Renyi maximal correlation between S\ and S2, 
i.e. 

/Omax = supE[5f(5i)/i(S , 2 )] (16) 
where the supremum is over all functions <?(•)> ^(") under which 

E[g(Si)] = E[h(S 2 )] = and E[ 5 2 (5i)] = E[h 2 (S 2 )] = 1. (17) 

For the bi-variate Gaussian memoryless source, condition (|15p reduces to (j!4j) because 
in this case /9 max is equal to p [151 Lemma 10.2, p. 182]. 

Remark 4.2. The necessary condition of Theorem 14.11 corresponds to the necessary 
and sufficient condition for the achievability of a distortion pair (Z?i,.D 2 ) when the 
source {(<Si,fc, <S2,fc)} is transmitted over a point-to-point AWGN channel of input power 
constraint P\ + P 2 + P\fP\Pi (see ©). This relation is not a coincidence. The proof of 
Theorem 14.11 (see Appendix [C]) indeed consists of reducing the multiple-access problem 
to the problem of transmitting the source {(Si,fc, S 2) k)} over an AWGN channel of input 
power constraint P\ + P 2 + P\fP\Pi- 

We now specialize Theorem l4.1l to the symmetric case. We combine the explicit form 
of the rate-distortion function in (jlOf) with (|14|) and substitute (D,D) for (Z?i,.D 2 ) to 
obtain: 



Corollary 4.1. In the symmetric case 

' 2P{\ +,.; + A' •"" T fc ly"- 7^7 

D*(o-*,p,P,N) > { 



CT 2P(l+p)+Af A 1 " 1 ' 1- 



2 / (1- P 2)JV , j> p 



10 



Corollary 14.11 concludes the section on the necessary condition for the achievability of 
a distortion pair (Di,D 2 ). We now compare this necessary condition to several sufficient 
conditions. The first sufficient condition that we consider is based on conventional 
source-channel separation. 



4.2 Source-Channel Separation 

As a benchmark we now consider the set of distortion pairs that are achieved by combin- 
ing the optimal scheme for the corresponding source-coding problem with the optimal 
scheme for the corresponding channel-coding problem. 

The corresponding source-coding problem is illustrated in Figure HI The two source 
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Figure 4: Distributed source coding problem for a bi-variate Gaussian source. 



components are observed by two separate encoders. These two encoders wish to describe 
their source sequence to the common receiver by means of individual rate-limited and 
error-free bit pipes. The receiver estimates each of the sequences subject to expected 
squared-error distortion. A detailed description of this problem can be found in 
The associated rate-distortion region is given in the next theorem. 

Theorem 4.2 (Oohama [5]; Wagner, Tavildar, Viswanath |6j). For the Gaussian two- 
terminal source coding problem (with source components of unit variances) a distortion- 
pair (D\,D2) is achievable if, and only if, 

(Ri,R 2 ) g ftipi) n n 2 (D 2 ) n n sum (D 1 ,D 2 ), 



where 



fti(£>i) 

K2(D 2 ) 



{R11R2) 
(ill, -R2) 
(ill, R2) 



Ri > l^g+ 



Ro > - lOE 



(1- p 2 (l-2~ 2R2 )) 



Di 

— (l-p 2 (l 

£>2 



-2Ri 



)) 



Ri + R2>- logj 



(l-p 2 )P(D 1 ,D 2 ) 
2D X D2 



with 



/3(D 1 ,D 2 ) = 1 + 4/1 + 



^P 2 D X D2 



2\2 ' 



(l-p 



The distortions achievable by source-channel separation now follow from combining 
Theorem 14.21 with the capacity of the Gaussian multiple-access channel (see e.g. |16pi7|). 
We state here the explicit expression for the resulting distortion pairs only for the 
symmetric case. 
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Corollary 4.2. In the symmetric case, a distortion D is achievable by source- channel 
separation if, and only if, 



2 V /N(N + 2P(l-pi)) 
~° 2P + N 

We next consider several combined source-channel coding schemes. The first scheme 
is an uncoded scheme. 

4.3 Uncoded Scheme 

In this section we consider an uncoded transmission scheme, which, as we show, is 
optimal below a certain SNR-threshold. 

The uncoded scheme operates as follows. At every time instant k, Encoder i £ 
{1, 2} produces as channel input X^ k a scaled version of the time-fc source output Si t k- 
The corresponding scaling is such that the average power constraint of the channel is 
satisfied. That is, 

X Zk = \[^Si,k fc€{l,2,...,n}. 

Based on the resulting time-k channel output Y k , the decoder then performs an MMSE 
estimate Sf k of the source output Si t k, i £ {1, 2}, k £ {1,2, ... , n}. That is, 

Sf >k = E[S ijk \Y k ] fc€{l,2,...,n}. 

The expected distortions (Df, D%) resulting from this uncoded scheme as well as the 
optimality of the scheme below a certain SNR-threshold are given in the following 
theorem. 

Theorem 4.3. The distortion pairs (D^,D^) resulting from the described uncoded 
scheme are given by 

Dl = J (l^±£ ^ = g2 (1-P 2 )^ , (18) 

Pi+P 2 + 2py/F\F^ + N 2 P l + P 2 + 2p^T\P^ + N V ' 

These distortion pairs are optimal, i.e., lie on the boundary of the distortion region, 
whenever 



P 2 (l-p 2 ) 2 {P 1 + 2p V r P[P 2 ) <Np 2 (2P 2 (l-p 2 )+N). (19) 



Proof. The evaluation of (D^D^) leading to (|18p is given in Appendix [Dl Based on 
the expressions for D" and D 2 the optimality of the uncoded scheme now follows from 
verifying that for all P\, P 2 and satisfying (I19p the corresponding distortion pair 
(DfjD^) satisfies the necessary condition (|14p of Theorem 14 . 1 1 with equality. To verify 
this, one can first verify that for all Pi, P 2 and N satisfying (|19p we have (D^D^) G 
9*. " □ 
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Remark 4.3. The optimality of the uncoded scheme can also be derived in a more 
conceptual way. To see this, denote by ^mac(c 2 i P-, P\, P2-, N) the distortion region 
for our multiple-access problem, and by ^ptp(<7 2 , p, P, N) the distortion region for the 
point-to-point problem of Section The optimality of the uncoded scheme for the 
multiple-access problem now follows from combining the following three statements: 

A) 

^mac (a 2 ,p,P 1 ,P 2 ,N) C ^p TP [a 2 , p ,P 1 + P 2 + 2p/P^, jv) . 
Statement A) is nothing but a restatement of Theorem 14.11 and Remark 14.21 



B) For the point-to-point problem of Section [3] with power constraint P = P\ + P2 + 
c lp\[P\Pi, let (Di,D%) be a distorion pair resulting from the uncoded scheme of 
Section [3~3l If 

— < T(Di,a ,p), 

where V is the threshold function defined in (|12p . then (D\, D2) lies on the bound- 
ary of ^ptp(<t 2 , p, P1 + P2 + Zp^/PJ^, N). 

Statement B) follows immediately by Proposition 13.11 



C) Let {piipi, (3), D2 (a, (3)) be the distortion pair resulting from the uncoded scheme 
for the point-to-point problem, and let (D±, D2) be the distortion pair resulting 
from the uncoded scheme for the multiple-access problem. Then, if 




then 

{DUa,P),D 2 1 (a,f3)) = {D u 1 ,D%). 



Statement C) follows since in the multiple-access problem, the channel output 

Y k = aS 1>k + (3S 2 ,k + Zk, 
resulting from the uncoded scheme mimics the channel output of the uncoded scheme 
for the point-to-point problem with power constraint P = P\ +P2 -\-1p\fP\Pi. Thus, 
while in the multiple-access problem the encoders cannot cooperate, the channel 
performs the addition for them. And since the reconstructors are the same in the 
multiple-access problem and the point-to-point problem, the resulting distortions 
are the same in both problems. 



Combining Statements A), B) and C), gives that if 

P1 + P2 + 2p^T\P 2 



y <T(Dl^,p), (20) 
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then (Df, L>2 ) lies on the boundary of ^mac(o~ 2 , p, Pi,P2,N), i.e., the uncoded scheme 
for the multiple-access problem is optimal. The threshold condition (|19p now follows 
by (|20f) and from substituting therein the value of by its explicit expression given 
in (USD. 



As a special case of Theorem 14.31 we obtain: 
Corollary 4.3. In the symmetric case, 

v ,H ' ' ' 2P(l + p)+N' 



P 

for all — < — 
1 N ~ 1 



(21) 



Moreover, for all SNRs below the given threshold, the minimal distortion D*(a 2 , p, P, N) 
is achieved by the uncoded scheme. 

The upper and lower bounds that result on D*(o~ 2 , p, P, N) from our derived neces- 
sary conditions and sufficient conditions are illustrated in Figure [5] for a source of cor- 
relation coefficient p = 0.5. For the SNRs below the threshold of (|2ip (marked by the 
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p = 0.5 


\ \ source-channel separation 




uncoded 




lower bound 



2 , 3 

P/N 



Figure 5: Upper and lower bounds on D*(a 2 , p, P, N) for a source of correlation coeffi- 
cient p = 0.5. 

dashed line) the uncoded approach performs significantly better than the separation- 
based approach. However, for SNRs above the threshold of (|2ip the performance of the 
uncoded scheme gets successively worse. By the expressions in (|2ip . we obtain that in 
the symmetric case 

hm D? = o 2 ±^, i€{l,2}. (22) 

P/N^oc 2 
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That is, as P/N — ► oo the distortion does not tend to zero. The reason is that as 
the noise tends to zero, the channel output corresponding to the uncoded scheme tends 
to aSi + (3S2, from which Si and S2 cannot be recovered. 



4.4 Vector-Quantizer Scheme 

In this section, we propose a coding scheme that improves on the uncoded scheme at 
high SNR. In this scheme the signal transmitted by each encoder is a vector-quantized 
version of its source sequence. The vital difference to the separation-based scheme is 
that the vector-quantized sequences are not mapped to bits before they are transmit- 
ted. Instead, the vector-quantized sequences are the channel inputs themselves. This 
transfers some of the correlation from the source to the channel inputs with the chan- 
nel inputs still being from discrete sets, thereby enabling the decoder to make distinct 
estimates of Si and of S2. For this scheme, we derive the achievable distortions and, 
based on those and on the necessary condition of Theorem 14. 1\ deduce the high SNR 
asymptotics of an optimal scheme. 

The structure of an encoder of our scheme is illustrated in Figure First, the 



VQ 

rate-it^ 




Figure 6: Encoder of Vector-Quantizer Scheme. 

source sequence Sj is quantized by an optimal rate-i?j vector-quantizer. The resulting 
quantized sequence is denoted by U*. For its transmission over the channel, this 
sequence is scaled so as to satisfy the average power constraint of (jl|). That is, the 
channel input sequence Xj is given by 



Based on the channel output Y resulting from Xi and X2, the decoder then estimates 
the two source sequences Si and S2- It does this in two steps. First, it tries to recover 
the two transmitted sequences and U?; from the channel output sequence Y by 
performing joint decoding that takes into consideration the correlation between two 
transmitted sequences \J\ and U?;. The resulting decoded sequences are denoted by 
Ui and U2 respectively. In the second step, the decoder performs approximate MMSE 
estimates Sj, i G {1,2}, of the source sequences Si based on Ui and TJ2, i.e. 

Si = TiiUi + 7i2U 2 , 
« E[Si|Ui,U 2 

A detailed description of the scheme is given in Appendix [El 

The distortion pairs achieved by this vector-quantizer scheme are stated in the 
following theorem. 
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Theorem 4.4. The distortions achieved by the vector- quantizer scheme are all pairs 
(Di,D2) satisfying 



D x > a z 2 

-2R 2 



2^ Rl i-p 2 (i-2- 2 ^; 
1-p 2 



1 -p 2 (l- 2~ 2R i ' 



1-p 2 



where the rate-pair [Rx^Rz) satisfies 



2 bI \ N(l - p 2 ) 
and where 

p = py/(l-2- 2R i)(l-2- 2R 2). (26) 

Proof. See Appendix [El □ 

Remark 4.4. The coefficient p corresponds to the asymptotic average correlation coef- 
ficient between two time-/c channel inputs Xxk an d -X2 fc- 

Based on Theorem 14.41 we now derive two more results: we show that for the 
symmetric version of our problem, source-channel separation is suboptimal also at high 
SNR, and we determine the precise high-SNR asymptotics of an optimal scheme. We 
begin with the sub-optimality of source-channel separation. To this end, we restate 
Theorem 14.41 more specifically for the symmetric case. 

Corollary 4.4. In the symmetric case 

D*(a 2 ,p,P,N)<a 2 2- 2R 1 ^ 2 > 



1- p 2 {l-2- 2R ) 2 ' 
where 

R<ho g J 2P ^ 1 + ^- 2 ^ + N 



4 &i V N{1- p 2 {l-2~ 2R ) 2 ) 

By comparing the achievable distortion of the vector-quantizer scheme, in Corollary 
with the achievable distortion of the separation-based scheme, in Corollary 14.21 we 
obtain: 

Corollary 4.5. In the symmetric case with p > 0, source- channel separation is subop- 
timal for all P > 0. 
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We turn to the high-SNR asymptotics of an optimal scheme. To this end, let 
(D^,D 2 ) denote an arbitrary distortion pair resulting from an optimal scheme. For a 
subset of those distortion pairs, the high SNR behavior is described in the following 
theorem. 

Theorem 4.5 (High-SNR Distortion). The high-SNR asymptotic behavior of{D\,D 2 ) 
is given by 

A™ N D x D 2 =a (1-p), 

provided that D\ < a 2 and D 2 < a 2 , and that 

N N 

lim = and lim = 0. (27) 

N^oPxDl N^oP 2 D* 2 1 ; 

Proof. See Appendix [FJ □ 

We restate Theorem 14 . 5 1 more specifically for the symmetric case. Since there D\ = 
D* 2 = D*(a 2 ,p,P,N), condition (|27|) is implicitly satisfied. Thus, 

Corollary 4.6. In the symmetric case 



lim J^D*(a 2 ,p,P,N)=a 2 J^—P. (28) 
P/N^oo V 1 V V 2 

Remark 4.5. Corollary 14.61 can also be deduced without Theorem 14.51 by comparing 
the distortion of the vector-quantizer scheme in Corollary 14.41 to the lower bound on 
D*(a 2 ,p,P,N) in Corollary SZQ 

To get some understanding of the coefficient on the RHS of (|28p . let us first rewrite 
(1281) as follows 



Next, let us compare this asymptotic behavior to that of two suboptimal schemes: the 
best separation-based scheme and the suboptimal separation-based scheme that com- 
pletely ignores the source correlation, i.e., the best scheme where the transmitters and 
the receiver treat the two source components as if they where independent. Denoting 
the distortion of the best separation-based scheme by .Dsb and the distortion of the 
scheme that ignores the source correlation by -Die; gives 



2. N(l-p 2 ) _ A n 2 IN P 



£>SB~^Y \ p and Z?ic~^y— , as 

The asymptotic expression for D^b follows by Corollary 14.21 and the asymptotic expres- 
sion for Die follows from combining the rate-distortion function of a Gaussian random 
variable, see e.g. [18, Theorem 13.3.2, p. 344], with the capacity region of the Gaussian 
multiple-access channel, see e.g. [TBI Section 14.3.6, p. 403]. 
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The asymptotic behavior can now be understood as follows. The denominator un- 
der the square-root corresponds to the average power that the scheme under discussion 
produces on the sum of the channel inputs Xi k + -^2,fc- In the two separation-based 
approaches this average power is 2P, and in the vector-quantizer scheme this average 
power is 2P(1 + p) as P/N — > oo. The numerator under the square-root consists of the 
noise variance N multiplied by a coefficient reflecting the gain due to the logical ex- 
ploitation of the source correlation. For the scheme ignoring the source correlation this 
coefficient is, by definition of the scheme, equal to 1, i.e., no gain, whereas for the best 
separation-based scheme and for the vector-quantizer scheme this coefficient is equal to 
I- p 2 . The means by which this gain is obtained in the best separation-based scheme 
and in the vector-quantizer scheme are fundamentally different. In the separation- 
based scheme the gain is achieved by a generalized form of Slepian-Wolf coding (see 
[5]), whereas in the vector-quantizer scheme the gain is achieved by joint-typicality de- 
coding that takes into consideration the correlation between the transmitted sequences 

and U2 (see Theorem 14. 4p . The corresponding advantage of the vector-quantizer 
scheme is that by performing the logical exploitation only at the receiver, it addition- 
ally allows for exploiting the source correlation in a physical way, i.e., by producing a 
power boost in the transmitted signal pair. 

4.5 Superposition Approach 

The last scheme of this paper is a combination of the previously considered uncoded 
scheme and vector-quantizer scheme. One way to combine these schemes would be by 
time- and power-sharing. As stated in Remark 12. 11 this would result in a convexification 
of the union of the achievable distortions of the two individual schemes. In this section, 
we instead propose an approach where the two schemes are superimposed. In the 
symmetric case, this approach results in better performances than time- and power- 
sharing, and for all SNRs, the resulting distortion is very close to the lower bound 
on D*(a 2 , p, P, N) of Corollary 14.11 We also point out that for the simpler problem 
of transmitting a univariate memoryless Gaussian source over a point-to-point AWGN 
channel subject to expected squared-error distortion, a similar superposition approach 
was shown in [19J to yield a continuum of optimal schemes. 

The superimposed scheme can be described as follows. The channel input sequence 
Xj produced by Encoder i, i E {1, 2}, is a linear combination of the source sequence Sj 
and its rate-i?j vector-quantized version U*. That is, 



where the sequence U| is obtained in exactly the same way as in the vector-quantizer 
scheme, and where the coefficients a{ and Pi are chosen so that the sequence Xj satisfies 
the power constraint (j4]), and so that the receiver can, with high probability, recover 
the transmitted codeword pair (U^U^). As we shall see, these two conditions will be 
satisfied as long as ctj and 0i satisfy to within some e's and <5's 



Xj = aiSi + (3iU* 



% 1 



(29) 




a 





(30) 



(For a precise statement see Appendix . 
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From the resulting channel output Y = Xi + X2 + Z, the decoder then makes a 
guess (Ui,U2) of the transmitted sequences (U^U^). This guess is obtained by joint 
typicality decoding that takes into consideration the correlation between UJ, TJ 2 , Si 
and S2. From the sequences Ui, U2, and Y, the decoder then computes approximate 
MMSE estimates Si and S2 of the source sequences Si and S2, i.e., 

Si = TiiUi + 7i 2 U 2 + 7*3 Y « e {1,2}, (31) 

where the coefficients 7^ are chosen such that Sj ~ E[SjY, Ui, Uj] . To state the 
explicit form of coefficients 7^, define for any rate pair i?2) 5 where E4 > 0, the 
3x3 matrix K( J Ri,i? 2 ) by 



K(R U R 2 ) = ki2 k 22 k 23 , (32) 



where 

kn = ,7 2 (l-2- 2i?1 ) 

k 12 = a 2 p(l-2~ 2R t)(l-2- 2R *) 

ki3 = («i + a2p)kn + /3 2 ki2 
k 22 = a 2 (l-2- 2 ^) 

k23 = ("2 + @2 + "lP)k 2 2 + /3lki2 

k33 = "iO" 2 + 2ai/?ikn + 2oi\ct 2 po- 2 + 2ai/3 2 /?k 2 2 + Pfkn + 2/3ia 2 pkn 
+ 2/3i/3 2 ki2 + 2a 2 (3 2 k 22 + ajja 2 + (3%k 22 + iV. 

The coefficients 7^ are then given by 



(i?i,i? 2 ) Q 2 i€ {1,2}, (33) 





where 

en = kn 
C12 = pk 22 

C13 = (ai + a 2 p)a 2 + /?ikn + (3 2 pk 22 
C21 = pkn 
C22 = k22 

C23 = ("2 + aip)a 2 + /Sipkn + (3 2 k 22 . 

The distortions achieved by the superimposed scheme are now given in the following 
theorem. 

Theorem 4.6. The distortions achieved by the superposition approach are all pairs 
(Di,D 2 ) satisfying 

Di> a 2 - jncn - j i2 c i2 - j i3 c i3 i € {1, 2}. 
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where the rate-pair (Ri,R 2 ) satisfies 



1 f ffknjl -f?) + N' \ 

^<2 log2 v inr^T) — J 

^ 2 < 2 bg2 I JV^j J 

Ri + R 2 < - lo g2 



2 b2 V ^'(1 " P 2 ) 

for some ct\, a 2 , 0i, and 2 satisfying (f30j) and where 

N' = a\vi + a\v 2 + 2a x a 2 v z + AT, (34) 

where 

v\ = a 2 - (1 - aip) 2 kn - 2(1 - aip)aiki 2 - a 2 k 2 2 
v 2 = a 2 - (1 - a 2y o) 2 k 22 - 2(1 - a 2 p)a 2 V\ 2 - a\V\\ 

u 3 = pa 2 - ((1 - oi/5)(l - a 2 /5) + aia 2 )ki 2 - (1 - ai/5)a 2 kn - (1 - a 2 p)a 1 k 22 , 

with 

0[ = ai(l — a\p) + 0\ + a 2 a 2 (35) 
02 = a 2 (l — a 2 p) +/? 2 + aiOi, (36) 

and u>ii/i 

a = 2 -2^ l(1 _ 2 -2 fi2) 

1 (i _ 2-2R2) _ 2pV(l " 2- 2i?1 )(l " 2- 2fi 2) +p 2 (l - 2~ 2R i) ' 

a = P 2-^(l-2-^) 

2 (1 - 2- 2 *i) - 2pV(! " 2- 2i?1 )(l " 2-2R 2 ) +/5 2 (1 _ 2 -2i?2) ' 

Proof. See Appendix [Gj □ 

In the symmetric case where P\ = P 2 = P, R± = R 2 = R and where ot\ = a 2 = a 
and 0x = 02 = /5, the matrix K(R,R) and the coefficients 7^ reduce to 

ki = a 2 (l - 2- 2R ) 
k 2 = a 2 p(l - 2- 2R ) 2 



and 



ki 


k 2 


k 3 


k 2 


ki 


k 3 


k 3 


k 3 


ki 



K(R,R) = k 2 ki k 3 where 

k 3 = (<* + £ + ap)ki + /3k 2 
k 4 = 2ac 3 + 20k 3 + AT, 



/ ci \ ci = ki 

= K _1 (i2,i2) [ c 2 where c 2 = pky 

\ c 3 / c 3 = (aa 2 + /3k 1 )(l + p). 

Thus, in the symmetric case Theorem 14.61 simplifies as follows. 
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Corollary 4.7. With the superposition approach in the symmetric case we can achieve 
the distortion 



infer 2 - 71C1 - 7 2 c 2 - 73C3, 
where the infimum is over all rates R satisfying 

1 ( 2/3*^(1 + p) + N' 

i?< 4 log2 V — Mi - p 2 ) — 

for some a and (5 satisfying 



a G 



P 



and j3 



! P-a 2 a 2 2~ 2R 
(J 2( 1 _ 2 -2i?) 



a, 



(39) 



and where 



and 



^ / = 2a 2 (^i + ^ 3 )+iV, 



with 



v\ = a 2 - 35- 

1 — p z 



-4R 



f 3 = 0- p 



l-p 2 ' 



To conclude our main results we have illustrated in Figure [7] all presented upper 
and lower bounds on D*(o~ 2 , p, P, N). 



5 Summary 

We studied the power versus distortion trade-off for the distributed transmission of a 
memoryless bi-variate Gaussian source over a two-to-one average-power limited Gaus- 
sian multiple-access channel. In this problem, each of two separate transmitters ob- 
serves a different component of a memoryless bi-variate Gaussian source. The two 
transmitters then describe their source component to a common receiver via a Gaus- 
sian multiple-access channel with average-power constraints on each channel input se- 
quences. From the output of the multiple-access channel, the receiver wishes to recon- 
struct each source component with the least possible expected squared-error distortion. 
Our interest was in characterizing the distortion pairs that are simultaneously achiev- 
able on the two source components. These pairs are a function of the power constraints 
and the variance of the additive channel noise, as well as of the source variance and of 
the correlation coefficient between the two source components. 

We first considered a different (non-distributed) problem, which was the point-to- 
point analog of our multiple-access problem. More precisely, we studied the power 
versus distortion trade-off for the transmission of a memoryless bi-variate Gaussian 
source over the AWGN channel, subject to expected squared-error distortion on each 
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Figure 7: Upper and lower bounds on D*(a 2 ,p, P, N) for a source of correlation coeffi- 
cient p = 0.5. 

source component. For this problem, we determined the set of achievable distortion 
pairs by deriving the explicit expression for the rate-distortion function of a memoryless 
bi-variate Gaussian source. Moreover, we showed that below a certain SNR-threshold 
an uncoded transmission scheme is optimal. 

For the multiple- access problem we then derived: 

• A necessary condition for the achievability of a distortion pair (Theorem 14. ip . 
This condition was obtained by reducing the multiple-access problem to a point- 
to-point problem. The key step was to upper bound the maximal correlation 
between two simultaneous channel inputs by using a result from maximum cor- 
relation theory. 

• The optimality of an uncoded transmission scheme below a certain SNR-threshold 
(Theorem I4.3[) . In the symmetric case, this result becomes (Corollary 14. 3p 



D*(a 2 ,p,P,N) = a' 



P(l - p 2 ) + N 
2P(l + p)+N' 



for all 



P 

N 



< 



l-p 2 



The strength of the underlying uncoded scheme is that it translates the entire 
source correlation onto the channel inputs, and thereby boosts the received power 
of the transmitted signal pair. Its weakness is that it allows the receiver to re- 
cover only the sum of the channel inputs. 



A sufficient condition based on a "vector-quantizer" scheme (Theorem I4.4j) . The 
motivation behind this scheme was to overcome the weakness of the uncoded 
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scheme. To this end, rather than transmitting the source components in an 
uncoded manner, the scheme transmits a scaled version of the optimally vector- 
quantized source components (without channel coding). 

• The precise high-SNR asymptotics of an optimal transmission scheme, which in 
the symmetric case are given by (Corollary I4.6|) 



This result follows from the "vector-quantizer" scheme (Theorem 14. 4p and the 
necessary condition of Theorem 14.11 

• The suboptimality, in the symmetric case, of source-channel separation at all 
SNRs. This follows from comparing the best separation-based approach (Corol- 
lary I4.2j) with the achievable distortions from the "vector-quantizer" scheme 
(Corollary IP)) . 

• A sufficient condition based on a superposition of the uncoded scheme and the 
vector-quantizer scheme (Theorem I4.6p . In the symmetric case this superposition 
approach was shown to be optimal or close to optimal at all SNRs. 

The presented sufficient conditions indicate that for the efficient exploitation of the 
source correlation it is necessary not only to exploit the source correlation in a logical 
way, e.g. by Slepian- Wolf-like strategies, but to additionally exploit the source corre- 
lation in a physical way. In the considered schemes, this is done by translating the 
source correlation onto the channel inputs. The logical exploitation of the source cor- 
relation is then performed at the receiver-side, e.g. by joint-typicality decoding taking 
into consideration the correlation between the channel inputs. 
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A Proof of Theorem 13.11 

Theorem 13.11 gives the expression of the rate-distortion function i?5 1) 5 2 (D 1 , D 2 ). A 
single-letter expression of this function, in the form of an optimization problem, follows 
from [20], Theorem 2, p. 856] and is 

R Sl ,s 2 (Di,D 2 )= p min I(S 1 ,S 2 ;S 1 ,S 2 ). (40) 

Si,S 2 \S 1 ,S 2 ' 
E[(5i-Si) 2 ]<Di 

e[(s 2 -s 2 ) 2 ]<d 2 

Thus, to prove Theorem 13.11 it remains to solve (140 p for all distortion pairs (D\,D 2 ) G 
(0, a 2 ] x (0, a 2 ]. One way of doing this was presented in [14] . Here, we present an alter- 
native approach. The derivation is split in two parts. First we derive Rsi,s 2 {Di, D 2 ) 
for (D 1 ,D 2 ) G 9>i, and then for (D 1 ,D 2 ) G (0,cr 2 ] x (0, a 2 ] \ Before starting with 
the derivations, we remark: 

Remark A.l. The restriction to E [S 2 ] = E [S 2 ] = a 2 and p G [0,1] incurs no loss in 
generality in the evaluation of Rs l ,s 2 (Di, D 2 ) since the distortion region scales linearly 
with the source variance, and since the distortion region is the same for correlation 
coefficients — p and p. 

A.l R Su s 2 {D Xl D 2 ) for the Region & x 

For pairs (Di,D 2 ) in S>\ the evaluation of Rs 1 ,s 2 (R > i, D 2 ) is very simple. The pairs 
(D\, D 2 ) G *2)\ are pairs where the larger of the two distortions can be achieved from the 
description of the other source component without any additional information simply by 
exploiting the correlation between S\ and S 2 . For example, since 52 = (S 2 — pS±)+ pS±, 
from an optimally vector-quantized version of S± that yields a distortion D\ , a distortion 
D 2 = <7 2 (1 — p 2 ) + p 2 D\ can be achieved on 52 by picking the reconstruction 52 of 52 
as a scaled version of the reconstruction S\ of S\. Thus, a necessary and sufficient 
condition to achieve a distortion pair (D\, cr 2 (l — p 2 ) + p 2 D\) is 




By symmetry, the above argument also works when the roles of S\ and 52 reversed. 
Hence, 

R Sl , S2 ( D l , D 2 ) = max { R Sl {D x ) , R S2 (D 2 ) } 

= l log2 {'B~) H(D 1 ,D 2 )e^ l . (41) 

A. 2 R Slt s 2 (D 1 , D 2 ) for the Regions % and % 

To solve P0|) for (Di,D 2 ) G (0,cr 2 ] x (0, a 2 ] \ S>\ we propose the following algorithm. 
First, we scale the source component 52 by a factor c G M + , where M + is the set of 
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positive reals numbers, and then we decorrelate (S\,cS 2 ) to a pair (Vi,V 2 ), by means 
of the unitary linear transformation: 

V\ \ ( a\ —a 2 \ f 1 \ ( Si 



v 2 J \ a 2 ai j \ o c j y s 2 1 ' {l2) 

where (ai,a2) T , (— a 2 ,ai) T are normalized eigenvectors of the covariance matrix of 
(Si,cS 2 ). Finally, we apply reverse waterfilling with rate R on (Vi,V 2 ), as described 
in [HI pp. 347]. The resulting reconstruction of (Vi, V 2 ) is denoted by (Vj,t^). From 
(VI , V2) the reconstruction (Si,S 2 ) of (Si,S 2 ) is then obtained by 

I)- b U)- (43) 

where the matrix B is the reverse transformation of (1421). i.e. 



B = ( I ^ J )• (44) 



- y v _fl 2 a i 

The probability law of (Si, S 2 ,Si,S 2 ) that results from the algorithm is a solution 
to the optimization problem in (140 j) . This is stated more precisely in the following 
theorem. 



Theorem A.l. 

i) For every distortion pair (Di,D 2 ) £ (0, a 2 ] x (0,cr 2 ] \ S>\ there exists some rate 
R* £ M + and some scaling coefficient c* G M + such that the distortion pair 
resulting from the above algorithm is (Di,D 2 ). 

ii) The probability law of (Si, S 2 , Si, S 2 ) resulting from this algorithm is a solution 
to the optimization problem in (|40p . 
For every (D 1 ,D 2 ) G (0,a 2 } x (0,<r 2 ]\^i 
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R Sl ,s 2 (Di,D 2 ) 



1 



log 2 



det Kss 



det (Ks S - K^ s ) 



where Kgg is t/te covariance matrix of the random vector 
the above R* and c* . 

Proof. See Section IA.31 



(45) 

3]) corresponding to 
□ 



Based on Theorem IA.1} we now evaluate Rs 1 ,s 2 {Di, D 2 ) for all {D\, D 2 ) G (0, a 



(0, a ] \*3l\ by expressing Kgg in terms of (D\,D 2 ). This is done in the following lemma 

Lemma A.l. For {D 1} D 2 ) G (0,a 2 ] x (0,a 2 } \ 
is given by 



\, the covariance matrix K§g in (|45p 



K 



■ss 




if{D u D 2 )e® 2 



a 



D x ){o 2 
D 2 



D 2 ) 



if(D u D 2 ) G 



(46) 
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Proof. See Section IA.4I □ 
Combining Lemma lA.ll with Theorem IA.1I gives 

' ii(Di,D 2 )e® 2 

R SuS2 ( y D 1 ,D 2 ) = I / \ 

|iogd ? — a l - p) v ) x(n 1: D 2 )e@ 3 . 

(47) 

The complete expression of Rs 1: s 2 (Di, D 2 ) for all (Di,D 2 ) G (0,cr 2 ] x (0,cr 2 ] is now 
obtained by combining (|47[) with (|41 1) . □ 

To conclude this section, we briefly point out that the above described algorithm 
easily extends to multi-variate Gaussians with more than two components. 

Remark A. 2. The extension of the above algorithm to multivariate Gaussians with 
more than two components is straight forward. For a source with v components 
(Si, S 2 , . . . , S u ), the scaling needs to be applied to u — 1 components, e.g. each of 
the components S 2 , • • • ,S U is scaled with a respective coefficient c 2 , ... c u in M + . The 
tuple (Si, c 2 S 2 , . . . , c u S u ) is then unitarily decorrelated to the tuple (Vi, V 2 , . . . , V u ) on 
which reverse waterfilling is applied. The reconstructions (Si, S 2 , ... , S u ) then follow 
from (Vi, V 2 , . . . , V v ) by the reverse of the transformation that mapped (Si, S 2 , . . . , S u ) 
to (Vi, V 2 , . . . ,V U ). The corresponding extension of Theorem IA.1I follows easily from 
the proof in Section IA.3I 

A. 3 Proof of Theorem IA.1I 

To prove Theorem I A. II we show that for every distortion pair (Di,D 2 ) G (0, a 2 ] x 
(0, a 2 } \ 3>i there exists a rate R* G M + and a scaling coefficient c* G M + such that 
E [(Si — Si) 2 ] = Di, E\(S 2 - S 2 ) 2 ] = D 2 , and R* = R Sl ,s 2 (D 1 ,D 2 ). The proof is then 

completed by computing I (Si, S 2 ; Si, S 2 ) and showing that it is equal to the RHS of 

(BSD. 

We begin by introducing the distortion regions 3>(R) and & C (R). The region @l(R) 
is the set of all pairs (Di,D 2 ) that are achievable with rate R for the source pair 
(Si, S 2 ). Similarly, the region 3) C (R) is the set of all pairs (D%, D 2 ) that are achievable 
with rate R on the scaled source pair (Si, cS 2 ). The following two remarks state some 
properties of these two regions. 

Remark A. 3. The regions @(R) and S! C (R) satisfy the following properties: 

i) 3i (R) and @ C (R) are convex. 

ii) S'c(R) is a linearly scaled version of *3s(R). The scaling is in the dimension of 
the LVaxis and of factor c 2 . 

hi) For a source satisfying Var(S'i) = \/ar(S 2 ), the region S>(R) is symmetric with 
respect to the line Di = D 2 . 

Proof of Remark \A.3l Part i) follows by a time-sharing argument. Part ii) follows by 
showing that if a distortion pair (Di,D 2 ) is achievable with rate R on (Si, S 2 ), then also 
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(D\, c 2 Z?2) is achievable with rate R on (Si, CS2), and vice versa. This follows since if a 
reconstruction pair (Si,^) results in distortions (D\,D2) on (Si, £2), then the scaled 
reconstructions C1S2) result in distortions (D\,c 2 D2) on (Si,cS2). Part hi) follows 
since for (Si, S2) jointly Gaussian with same variances, the distribution between Si and 
S2 is perfectly symmetric. Hence, if with rate R the pair (D\, D2) = (a, b) is achievable, 
then also the pair (D\,D2) = (b, a) is achievable. □ 

Remark A. 4. The scaled reconstruction pair (Si, CS2), where (Si, S2) is the result from 
our algorithm at rate R, yields the expected distortion pair (D\,D2) of minimal sum 
D x + D 2 in %(R). 



Proof of Remark \A.4 . We denote by Aj and A2 the distortion on Vi and V2, i.e. 



A; 



(Vi-Vif i e {1,2}. 



By definition of the reverse waterfilling solution, the reconstruction pair (V\, V2) achieves 
the distortion pair (Ai,A2) on (Vi,V 2 ) of minimal sum Ai + A2 among all v&te-R 
achievable pairs (Ai, A 2 ). Since (Si, CS2) relates to (Vi, V2) by the same unitary trans- 
formation that relates (Si, 082) to (Vi,Vi), the sum of the distortions D\ + D2 on 
(Si,cS2) equals Ai + A2. Hence, if (Ai, A2) is the pair of minimal sum among all 
rate-i? achievable pairs on (Vi, V2), then D\, D2 is the pair of minimal sum among all 
rate-i? achievable pairs on (Si,cS2), i.e., in S> C (R). □ 

We are now ready to start with the proof of Theorem IA.ll 

Proof of Theorem \A.1[ We first prove the Parts i) and ii) together. To this end, we 
begin by arguing that for every boundary point (Di,D^) of @(R) that falls out of 
3>i and satisfies D\ < D2, there exists some c G (0, 1] such that our algorithm yields 
(Di,!^)- To begin, consider c = 1. For this case, the distortion pair resulting from 
our scheme is the boundary point of &(R) that satisfies D% = D2. This can be seen by 
noticing that for c = 1 the regions @(R) and @ C (R) coincide. Thus, by Remark IA.41 
the distortion pair (D\, D2) resulting on (Si, S2) is the one of smallest sum D\ + D2 in 
3>(R). This distortion pair is the point on the boundary of 3>(R) that satisfies D\ = D2, 
since, by Remark IA.31 the region @(R) is convex and symmetric with respect to the 
line satisfying D% = Z?2- 

Next, consider < c < 1. The key idea of our algorithm is illustrated in Figure 
[HI Subplot a) shows the distortion region S) C (R), and Subplot b) shows the distortion 
region S>(R). The distortion pair (D^D^) resulting from our algorithm on (Si,S2) 
and the corresponding distortion pair (DI, c 2 D^) on (Si, CS2) are marked with a dot in 
Subplot b) and Subplot a) respectively. By Remark lA. 41 the distortion pair (D\,c 2 D?2) 
is the point of smallest sum in @ C (R). Hence, graphically this point is characterized as 
the point on the boundary of 2> C (R) for which the straight line of slope —1 containing 
it has the smallest ordinat^]^. By this graphical characterization of (D\,c 2 D2) it can 
now be seen that the smaller c gets, i.e., the more S> C (R) is shrunk along the Z?2-axis, 



1 This characterization follows since for any £ G R the set of pairs (Di, -D2) with sum Di + D2 = £ 
is the straight line of slope —1 and ordinate f. Hence, if the ordinate £ of the straight line is minimized 
then the sum D\ + D2 is minimized. 
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the smaller the associated distortion Dl gets. And as c — > 0, the distortion D\ tends 
to the minimal value of o~ 2 2~ 2R . Thus, by the linear relationship between & C {R) and 
£}{R), and by the convexity of @(R), it follows that for c G (0, 1] our algorithm can 
achieve any (.Di,!^) on the boundary of @(R) for which a 2 2~ 2R < D\ < L>2- 

For c > 1, it can be shown by similar arguments that our algorithm can achieve 
any (Di,L>2) on the boundary of S>{R) for which o 2 2~ 2R < D2 < D\, i.e., for which 
D2 < Di < a 2 (l — p 2 ) + p 2 D\. Hence, for c G M + our algorithm can achieve any 
boundary point of 9{R) in (0, a 2 } x (0, a 2 } \ 9\. 

To complete the proof of Parts i) and ii), we now show that with the appropriate R, 
the boundary of 3>{R) can indeed cover any point in [0, a 2 ] x [0, a 2 ]\3$i, and that for each 
such boundary point of @(R) the rate-distortion function satisfies i?5 1) 5 2 (D 1 , D2) = R- 

Lemma A. 2. For every distortion pair {D\,D2) € (0, a 2 ] x (0,cr 2 ] \ @/\ } there exists 
some rate R* £ M + such that {D\,D2) is in the boundary of &>{R*). Furthermore, for 
each such boundary point 

R Sl ,s 2 (Di,D 2 ) = R*. 

Proof of Lemma \A.2k We know that each distortion pair (D\,D2) resulting from our 
algorithm at rate R lies on the boundary of S'(R), and, by (|42h and (I43p . is given by 

Dl = ofAi + a 2 A 2 , D* 2 = + 4 A 2- (48) 

c c 

Since the distortion pair (Ai, A2) results from reverse waterfilling, either both of the 
distortions are strictly and continuously decreasing in R, or one of them is constant 
while the other is strictly and continuously decreasing in R. Thus, since for every 
c G M + , the coefficients a\ and 02 are both non-zero and constant, both distortions 
Dl and D\ are strictly and continuously decreasing in R. Hence, for every e > 0, no 
boundary point {D\, D2) of 3){K) belongs to &{R — e), and therefore, every boundary 
point (Di,D 2 ) of @{R) satisfies 

R Sl ,s 2 (D 1 ,D 2 )=R. 

The proof of of Lemma IA.2I is now completed by recalling that for each R G M + the 
region Sl{K) is convex, and that for every fixed c G M + the corresponding boundary 
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point in 3>(R) is evolving continuously in R. Thus, since for R = the distortion 
region is given by S'(R) = (a 2 , a 2 ) and for R — > oo the distortion region is given by 
S>{R) = [0,cr 2 ] x [0, a 2 }, it follows that with the appropriate R G M + the boundary of 
!3{R) can cover any point in [0, a 2 } x [0, a 2 ]. □ 

We now turn to the proof of Part hi) . The proof consists of computing I(S\ , S 2 ; Si, S 2 ) 
and showing that it is equal to the RHS of (|45p . We use the shorthand notation V for 
(V 1 ,V 2 ,Vi,V 2 ) and S for (Si, S 2 , Si, S 2 ). Thus, 



B 
B 



V. (49) 



Since S and V are related by a linear transformation and since V is a zero-mean 
Gaussian vector, it follows that also S is a zero-mean Gaussian vector. Thus, 

1 T ( det K55 det K ss 



= s"» I det% )• (50) 

where is the covariance matrix of S. The determinant det in ()50p is now 
expressed in terms of K55 and K^. Since the reconstruction pair (5i,S 2 ) achieves a 
boundary point of @(R), it must satisfy the orthogonality principle and thus K s § = Kaa, 
where K s § denoted the cross-covariance matrix between (Si,S 2 ) and (Si,S 2 ). Hence, 

^=(^ fc). (51) 
V ^ss ^ss J 

Applying Schur's complement [22l pp. 21] to (fSTI) gives 

det K§§ = det K§§ • det (Kss - L%) . (52) 

Combining ([52]) with (i50j) . and using that the law of the (S\, S 2 , Si, S 2 ) at hand is a 
solution to the optimization problem in (|40p . gives 




i? Sl Sopi, Ai) = - log 2 I , det ^ r I , (53) 

for all (Di,D 2 ) G (O,^ 2 ] x (0,<r 2 ] \ ^. □ 



A. 4 Proof of Lemma IA.ll 

Lemma lA.ll expresses K^, in terms of (Di,D 2 ). To obtain the stated expression, we 
first use that K<^, = BKy^B T and begin by writing Kyy in terms of the variances of 
Vi and V 2 , and in terms of the distortions on those two components. To this end, we 
denote by Ai and A 2 the variances of Vi and V 2 , i.e. 

A i = Var(Fi) ie{l,2}. 
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By (142 p . the variances Ai and A2 are the eigenvalues of the covariance matrix of 
(Si, CS2), which are given by 



A 1 



A 2 



a 



a 



1 + c 2 - V / l-2c 2 (l-2 / o 2 )+c 4 



(54) 



1 + c 2 + y/1 - 2c 2 (l - 2p 2 ) +c 4 ) . 



The covariance matrix Ky^ can now easily be expressed in terms of Ai, A2, Ai, and A2. 
Since for every c G M + the expressions in (|54p yield Ai < A2 , the reverse waterfilling 
on (Vi,V2) satisfies 

f A 2 if < A 2 < Ai 



[ Ai if Ai < A 2 < A 2 . 
The corresponding covariance matrix Kyy is thus 



(55) 



K, 



w 



Ai - A 2 
A 2 - A 2 





A2-A2 



if < A 2 < Ai 



if Ai < A 2 < A 2 . 



(56) 



Based on (|56p . we now express K^g = BK^B T in terms of D\ and D2. To this end, 
we denote the set of all distortion pairs (D\,D2) resulting from < A2 < Ai by 8$ 
and the set of all distortion pairs resulting from Ai < A2 < A2 by S3. The expressions 
for and B8, and for K<^, in terms of D\ and D2, are now given in the following two 
lemmas. 



Lemma A. 3. For (Di,!^) G (0, a ] x (0, a ] \ the covariance matrix in ([35 
is given by 



K 



a 2 - D x pa 2 

pa 2 a 2 - D 2 
a 2 -D 1 

V^ 2 -^!)^ 2 -^) 



if(Dx,D 2 ) G 



if(D u D 2 ) G 



Proo/. See Section ES 

Lemma A. 4. T/ie regions si and S3 are given by 

£? = @2 and S3 = 

Proof. See Section IA.4.11 

Combining Lemma lA.31 and Lemma IA.4I yields Lemma lA.ll 



(57) 
□ 



□ 

□ 
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Proposition A.l. Every (D\, D2) m^iU^3 can be achieved with rate Rs 1 5 2 (jDj, D2) 
by optimally describing a linear combination of the sequences Si and 82- 



Proof. We prove Proposition lA.ll for the regions &\ and ^3 separately. For S>\ the proof 
follows immediately from Section lA.ll where it is shown that every (Di,!^) in ^1 is 
achieved with rate Rs lt s 2 (Di, -D2) either by optimally describing Si or by optimally 
describing S2. For £^3 the proof follows from combining (j56[) with Lemma lA.41 from 
which it follows that every (Di,!^) in £^3 is achieved with rate Rs lt S2 D2) by 
optimally describing the sequence of corresponding ^-components of (Si,S2). This 
sequence, by the definition of V2, is a linear combination of Si and S2. □ 

It remains to prove Lemma I A. 31 and Lemma lA.41 We begin with Lemma I A. 41 



A. 4.1 Proof of Lemma IA.4I 

Lemma lA.41 determines the sets srf and SB in terms of the distortions D\ and Z?2- To 
prove this lemma we first derive the expression for stf '. The expression for SB will then 
be deduced by noticing that and SB form a partition of (0, a 2 ] x (0, a 2 ] \ @\. 

The region s# is defined as the set of all {D\,D2) deriving from < A2 < \\. To 
translate this condition to an expression in terms of D\ and D2 , we express A2 in terms 
of D\ and Z?2- 

For < A2 < Ai however, the reverse waterfilling solution in (|55p yields Aj = A2 
such that (|4"5]l simplifies to 

D x = A 2 D 2 = \a 2 . (58) 

Thus, the region srf can be restated as the set of all pairs (Di,!^) G (0, a 2 } x (0, a 2 ] 
satisfying < D\ < X\ and D1/D2 = c 2 . Writing out Ai as in (f54"j) and substituting 
therein c 2 by D1/D2 yields 



2 



0<D2<(o- 2 (l-p z )-D 1 ) . (59) 

0" z — L>i 

On the other hand, substituting the RHS of (|54l) in D% = Ai leads to 

max D\ = cr 2 (l — p 2 ), 

where the maximum is obtained as c — > 00. Thus, the region =2/ can finally be restated 
as 



[D 1 ,D 2 ):0<D 1 <a 2 (l- p 2 ),0<D 2 < (a 2 (l-p 2 )-D 1 



a 



2 



i.e., $4 = @ 2 . 

We now turn to the evaluation of 8$. As can be verified by help of (|48j) the two 
sets srf and SB form a partition of (0, a 2 ] x (0, a 2 ] \ Thus, the region SB is given by 
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(0, a 2 } x (0, a 2 } \{sfU9x). Hence, 



(D 1 ,D 2 ):0<D 1 <a 2 (l-p 2 ), 



(a 2 (l-p 2 )-D 1 ) 



a 2 -D, 



<D 2 <a 2 (l-p 2 ) + p 2 D 1 ; 



a 2 {l - p 2 ) <D X < (J 2 , 
D.-a^l-p 2 ) 



<D 2 <a 2 (l- p^)+p z D 1 



i.e., 



9*. 



□ 



A. 4. 2 Proof of Lemma IA. 31 



Lemma lA.3l gives an explicit expression of Kgg for (D\,D 2 ) £ ^ an d for (Di,D 2 ) £ 
The derivations are based on the expression for in (|56p . Combining (|56p with 



K§§ = BK^B T gives 



K, 



'SS 



i 2 X 1 + a 2 X 2 -A 2 2isa(A 2 -Ai; 



ffl£a(A 2 -Ai) 

a|(A 2 -A 2 ) ^(A 2 -A 2 ) 
I V ^(A 2 -A 2 ) |(A 2 -A 2 ) 



i,(a 2 Ai + a 2 A 2 - A 2 ) 



if (Di,D 2 ) G ^ 



if (Di,D 2 ) e 



(60) 



For (Di,D 2 ) £ s/, we now express the variables in (|60p in terms of .Di and -D 2 . 
From (15811 it follows that 



A, = £>i 



and 



Do 



Furthermore, the coefficients a\ and a 2 are determined by a 2 + a 2 = 1 and 

2 2 C y 



1 - 2c 2 (l - 2p 2 ) + c 4 + (1 - c 2 )Vl-2c 2 (l-2 / o 2 )+c 4 



(61) 



(62) 



Combining (|6T|) . (|62j) and the expressions for Ai and A 2 in (j54|) with (|60j) . yields 

<7 2 - L>! p<7 2 



K 



'S'.S' 



pa 2 a 2 - D 2 



if (D U D 2 ) £ srf. 



(63) 



We now turn to the evaluation of Kgg for the region S3. For this region, no cal- 
culations are needed. It suffices to notice that since every optimal reconstruction pair 
(Si, S 2 ) satisfies the orthogonality principle, the main diagonal terms of the covariance 
matrix K§g are 



a 2 (A 2 - A 2 ) = a 2 - D x and %(A 2 - A 2 ) = a 2 - D 2 , 
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and that the anti-diagonal terms are both equal to the square-root of the product of 
the two main diagonal terms and thus are 



a\a 2 



(A 2 - A 2 ) = vV-£>i)(<t 2 --E>2). 



The covariance matrix K§§ for 38 is therefore 



a 2 - Di y /( a 2_ Dl){a 2_ D2) 

K»=| ! if (D 1: D 2 )e 

V(<7 2 -I>i)(a 2 -D 2 ) a 2 - D 2 



(64) 
□ 



B Proof of Proposition 13.11 

Proposition 13.11 pertains to the point-to-point problem of Section O in which the source 
pair {(Si kiSz k)} is to be transmitted over an AWGN channel. It states that for an 
achievable distortion pair (D\,D 2 ) for which the SNR of the channel satisfies P/N < 
Y(D\, a 2 , p), there exist a* ,[5* > such that 

D^(a*,(3*) < D 1 and D%(a* , f3*) < D 2 . 

The essence of Proposition 13.11 is that the uncoded scheme proposed in Section 13.31 
achieves every distortion pair (D\,D 2 ) in 3>\ U 3#3 with the least possible transmission 
power, i.e., with the P for which 

R Sl ,s 2 (Di,D2) = \log 2 + ^ 

In Proposition 13. 11 the condition (D\, D 2 ) E 3>\ U ^3 is merely expressed in form of the 
threshold T(Di, a 2 , p) on P/N. 

We start the proof by showing that the uncoded scheme indeed achieves every 
(Di,D 2 ) G fiU^ with the least possible transmission power, respectively at the 
smallest P/N. To this end, let <&(D 1 ,D 2 ) be the smallest P/N at which (D 1: D 2 ) is 
achievable, i.e. 

R Sl ,s^Du D 2 ) = 1 log 2 (1 + *(L>i, D 2 )) . 

We now show that for every [D\,D 2 ) G S>\ U ^3, there exist a* , (3* such that the distor- 
tions resulting from the uncoded scheme at P/N = fy(Di,D 2 ) are (Df(a, (3),D 2 {a 1 (3)) = 
(Di,D 2 ). To show this, we rely on Proposition IA.ll (p. l3Tj) and on the result of pQ. 
Proposition IA. II states for the corresponding source coding problem that if (Di,D 2 ) is 
in S#iU2#3 then i?5 lj 5 2 (D 1 , D 2 ) can be achieved by optimally describing a linear combi- 
nation of the sequences Si and S 2 . The result of [1] states that the minimum expected 
squared-error transmission of a Gaussian source over a AWGN channel is achieved by 
uncoded transmission. Thus, by combining Proposition IA.1I with the result of [1] and 
using that since {(S\ t k, S 2 ,k)} are jointly Gaussian, each of their linear combination 
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aSi + f3S 2 is also Gaussian, it follows that every distortion pair (D\, D 2 ) £ f 1 U f 3 is 
achieved at P/N = fy(Di, D 2 ), by sending over the channel 



X%(a, ff) = ^l a2(a2 + 2 pa p + p2) (*S ltk + (3S 2jk ) k E {1, 2, . . . , n}, 

with the appropriate a, (3 > 0. 

It remains to derive the threshold function V. To this end, first notice that for an 
arbitrary fixed D\ E [0, a 2 ], the smaller the associated D 2 gets, the larger fy(Di, D 2 ) 
becomes, i.e., for a fixed D\ the function ^(-Di, D 2 ) is decreasing in D 2 . Now, for every 
Di G [0,ct 2 ], let £j 2 (£>i) be the smallest £> 2 such that (Di,D 2 ) 6^U ^3. Then, for 
every D x £ [0, a 2 ] 

T(D 1 ,a 2 ,p) = ^(D 1 ,D 2 (D 1 )). 

Hence, it remains to evaluate ^(-Di, D 2 (D{)) for every D\ £ [0, a 2 ]. We have 

f (a 2 (l - p 2 ) - DA - 7 £ r if0<L>i <a 2 (l- / o 2 ), 
1 if J D 1 >a 2 (l-p 2 ). 

For Di > cr 2 (l - /? 2 ) it immediately follows that T(Di, a 2 ,p) = 00. For < D\ < 
a 2 (l — p 2 ) the value of ^(D\, D 2 (Di)), and hence the value of T(D\, a 2 , p) follows from 
solving 



1 

-log+ 



2 



■ log 2 1 + T7 > ( 66 ) 



2 oz \ iV 



\DxD2 - (pa 2 - ^{a 2 -D^a 2 -D 2 ) 
where we have used the shorthand notation D 2 for D 2 {D\). From (|65p . we now get 



pa 2 -yJ(a 2 -D 1 )(a 2 -D 2 ) = 0. 

Thus, (|B"6]) reduces to 



a 4 (l-» 2 ) P , . 

1 + T7- (67) 



£>iL>2 ^ 
which, by ()65[) . can be rewritten as 

P_ _ <y\l- p 2 )-2<j 2 D 1 {\- p 2 )+D 2 
N ~ D x (ct 2 (1- p 2 )- Di) 



(68) 



This is the threshold given in Proposition O for < D x < cr 2 (l - p 2 ). 

To conclude the proof, we justify the restriction to a, (3 > 0. This restriction is made 
because from the expressions for Df(a, [3) and D^ia, (3) it follows that it incurs no loss 
in performance. This is so, since p > 0, and thus the uncoded transmission scheme 
with the choice of (a, (3) such that a ■ (3 < yields a distortion that is uniformly worse 
than the choice of (\a\, \/3\), and every distortion pair achievable with a,(3<0, is also 
achievable with (\a\, \(3\). Thus, without loss in performance, we can limit ourselves to 
a,P>0. □ 
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C Proof of Theorem 14.11 



Theorem 14. 1 1 applies to the multiple- access problem without feedback. For this problem 
it gives a necessary condition for the achievability of a distortion pair (D\,D2). We 
begin with a reduction. 

To state the proof in more detail, we make the following reduction. 

Reduction C.l. There is no loss in optimality in restricting the encoding functions 
to satisfy 

E[X hk ] =0 for i G {1, 2}, and all fcgZ. (69) 

Proof. We show that for every achievable tuple (Di, D2, a 2 , a 2 , p, P\, P2, N), there ex- 
ists a scheme with encoding functions satisfying (|69j) that achieves this tuple. To 
this end, let (D\ , D2 , o 2 , a 2 , p, P\ , P2 , N) be an arbitrary achievable tuple. Further, let 
{/1 }) {/i™^}' {0^} be sequences of encoding and decoding functions achieving this 
tuple. If the encoding functions {f[ n ^}, {f% } do not satisfy ([69]) . then they can be 
adapted as follows. Before sending the codewords over the channel, the mean of the 
codewords is subtracted so as to satisfy (f69|) . And at the channel output this subtrac- 
tion is corrected by adding this term to the received sequence before decoding. □ 

In view of Reduction IC-H we restrict ourselves, for the remainder of this proof to 
encoding functions that satisfy (|69p . The key element in the proof of Theorem 14.11 is 
the following. 



Lemma C.l. Any scheme satisfying condition (I69p and the original power constraints 
flU), also satisfies 

1 - 

- J2 E K x i,k + X2,kf] <Pi + P2 + 1p\[P^Pi- (70) 
fc=l 

Proof. See Appendix IC.ll □ 

Based on Lemma IC.ll the proof of Theorem 14.11 is now obtained by relaxing the 
original problem as follows. First, the power constraint of ([4]) is replaced by the power 
constraint of (|70p . Then, under the power constraint of (|70|) . the two transmitters are 
allowed to fully cooperate. These two relaxations reduce the original multiple-access 
problem to a point-to-point problem where the source sequence {(<5i &, S^fc)} is to be 
transmitted over an AWGN channel of power constraint P± + P2 + 2p\fP\p2 and noise 
variance N. For this point-to-point problem, a necessary condition for the achievability 
of a distortion pair (D\,D2) follows by source-channel separation, and is 

Rs 1 ,S2[Dl,D2) < 2 lo S2 ( 1 + ^ )• (71) 

It is now easy to conclude that (I7ip is also a necessary condition for the achievability of 
a distortion pair (D\,D2) in the original multiple-access problem. This simply follows 
since (17ip is a necessary condition for the achievability of a distortion pair (D±,D2) in 
a relaxed version of the multiple-access problem. This concludes the proof of Theorem 
HH □ 
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C.l Proof of Lemma IC .11 

The key to Lemma IC. II is as follows: 

Lemma C.2. For any coding scheme with encoding functions of the form ([3]) that 
satisfy the power constraints dH) and condition (I69p of Reduction \C.1\ and where the 
encoder input sequences are jointly Gaussian as in §\§ with non-negative correlation 
coefficient p and equal variances a\ = a\ = a 2 (Reduction \2.1\) . any time-k encoder 
outputs Xi t f, and X 2: k satisfy 



E[X ljit X 2)fc ] < P JE 



Y 2 



v2 

A 2,fc 



(72) 



Proof. Lemma IC , 2 1 follows from two results from Maximum Correlation Theory. These 
results are stated now. 

Theorem C.l (Witsenhausen [5T]). Consider a sequence of pairs of random variables 
{(Wl A) ^2 k)}> where the pairs are independent (not necessarily identically distributed). 
Then, 



sup E 

(n) (n) 

9l >92 



5i n) (W 1 )^ ; (W 2 ) < sup Elg^W^k^k^k)} 

l<k<n 



(73) 



where the supremum on the LHS of ([73]) is taken over all functions : W 1 
satisfying 



aT ] (W 4 )' 



o 



l z e {1,2}, 



and the supremum on the RHS of ([73"]) is taken over all functions g-i t k 
satisfying 







sffi Wit) 



1 i€ {1,2}. 



Proof. See (2TJ Theorem 1, p. 105]. 



□ 



Lemma C.3. Consider two jointly Gaussian random variables W\k an d W 2 ,k with 
correlation coefficient p/.. Then, 

sup E[gi,k(W 1>k )g 2 ,k(W2,k)] = \Pkl 

where the supremum is taken over all functions gij. : R — * M, satisfying 

E[9iA w i,k)] = E[{gi,k(W i;k )) 2 ] =1 ie {1,2}. 
Proof. See [El Lemma 10.2, p. 182]. □ 
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Lemma fC. 21 is now merely a consequence of Theorem I C . 1 1 and Lemma fC . 31 applied to 
our setup. To see this, substitute Wi and W2 by the source sequences Si and S2, and 
let the functions g^ (•) and g 2 (•) be the encoding sub- functions that produce the time- 
k channel inputs Xi ik and X 2 , k , i-e., g\ (Si) = Xi )k . Then, for every k G {1,2, ... , n}, 



E[-X"i,fe-^2,fe] 



X ik 



< sup E 

(n) (n) 

9i ,92 



g ( [ l \s 1 )gi n \s,) 



a) 

< sup E[gi,k{Si,k)g2M s 2,k)] 

l<k<n 

9\,k,92,k 

b) 

<P, 



(74) 



where a) follows from Theorem 10.11 and b) follows from Lemma IC.3I and from our 
assumption that p > (Reduction 12 . X [) . Thus, for every time k, 



E[X 1>k X 2 , k ]<pJE 



X h 



v2 

A 2,fc 



which is the bound of Lemma IC.2I 

Using Lemma IC.21 we can now prove the bound of Lemma I C. II as follows: 



(75) 
□ 



i £ e [( Xl , k + x 2 , k f] = \ ^ e [xl k ] + i £ e [xl k ] + 2 l -Y, mi,*x 2 , k ] 

k=l k=l 

1 n 

<P l + P 2 + 2- VE[I 1)fc I 2)t ] 



k=l 



k=l 



k=l 



<A + P 2 + 2p-WE 



n 

b ) 1 
< Pi + P 2 + 2p- 
n 



k=\ 



X h 



x ik 



n n 
\ k=l \ k=l 



2 

2,k 



<Pi + P 2 + 2p-^P x ^f^P 2 
n 



Pi + P 2 + 2p^/F\P 2 , 



(76) 



where Inequality a) follows by Lemma IC.2I and from our assumption that p > 0, and 
where Inequality b) follows by Cauchy-Schwarz. This concludes the proof of Lemma 

ELH □ 



D Distortions (D^, D%) of the Uncoded Scheme 

The expression for Df, i £ {1, 2}, is obtained as follows 

1 11 



k=l 
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n 



2E 



1 

n 



2 

QU 



+ E 



i.k 



E [<Sf,fc] 



E[n 2 ] 



C) ^2 2 

= (T — a 



P 1 + 2p^7\P^ + p 2 P 2 
Pi + 2p/PT7^ + P 2 + N 

Pi(l-p 2 ) + iV 



P l +P 2 + 2p^/l\P 2 + N' 

where in a) we have used that Sf k = E[Sik\Yk] satisfies the Orthogonality Principle; in 
b) we have used the explicit form of the conditional mean for jointly Gaussians 

qu _ pre ivi _ E[ffi,fcYfc] v 
and in c) we have used the calculation 



:E[g 1|fc y fc ])- 

EK] 



a 2 (VK + pV^y 

P 1 +P 2 + 2p^/7\P 2 ~ + N' 



□ 



E Proof of Theorem 14.41 

In this appendix we analyze the distortions achievable by the vector-quantizer scheme 
that was presented in Section 14.41 To start, we give a thorough description of the 
corresponding coding scheme. 



E.l Coding Scheme 

Fix some e > and rates R\ and R 2 . 

Code Construction: Two codebooks C\ and C 2 are generated independently. Code- 
book d, i G {1,2}, consists of 2 nR - codewords {U<(1), Uj(2), . . . , U i (2 n - R *)}- The code- 
words are drawn independently uniformly over the surface of the centered R n -sphere 
Si of radius = \/ na 2 (l — 2~ 2Ri ). 

Encoding: Based on the observed source sequence Sj each encoder produces its chan- 
nel input Xj by first vector-quantizing the source sequence Sj to a codeword U* e C« 
and then scaling U* to satisfy the average power constraint. To describe the vector- 
quantizer precisely, denote for every w, v G M. n where neither w nor v are the zero- 
sequence, the angle between w and v by <(w, v), i.e. 

cos<(w,v) 4 (77) 
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Let 

F{si,Ci) = [ui e C t : Vl- 2^(1 - e ) < cos<(s J ,U l ) < ^1-2-2^(1 + e)} . (78) 

If J-(si,Ci) 0, the vector-quantizer output U* is the codeword Uj(j') € J-(si,Ci) 
that minimizes | cos <(uj(j), Sj) — \/T— 2~ 2 - R » |, and if ^"(sj, Cj) = the vector-quantizer 
output U* is the all-zero sequence. Thus, 



argmin UieCi: cos <(u,(j), S,) - Vl - 2" 2 ^ if ^( Si , Cj) / 0, 

otherwise. 

More formally, U* should be written as U*(Sj,Cj), but we shall usually make these 
dependencies implicit. The channel input is now given by 



X i = a i U? where a , = J — -A_ * e {1,2}. (79) 



Since the codebook Cj is drawn over the centered R n -sphere of radius r\ = y a 2 {l — 2 2Ri 
each channel input Xj satisfies the average power constraint individually. 

Reconstruction: The receiver's estimate (Si, S 2 ) of the source pair (Si, S2) is derived 
from the channel output Y in two steps. First, the receiver makes a guess (Ui,U 2 ) of 
the pair (U*, U 2 ) by choosing among all "jointly typical pairs" (Ui, U2) £ C\ x C 2 the 
pair whose linear combination aiUi + a 2 U 2 has the smallest distance to the received 
sequence Y. More precisely, 

(Ui,U 2 )= argmin ||Y - ( Ql Ui + a 2 U 2 )|| 2 , (80) 
(Ui,u 2 )eCixC 2 : 

p— cos <(ui,U2)|<7e 

where 

p = p^/(l-2- 2fl i)(l-2- 2fl 2). 

If the channel output Y and the codebooks C\ and C 2 are such that there does not exist 
a pair (Ui,U 2 ) G C\ x C 2 that satisfies 

|/5-cos<(ui,u 2 )| < 7e, (81) 

then Ui and U 2 are chosen to be all-zero. 

In the second step, the receiver computes the estimates (Si,S 2 ) from the guess 
(Ui,U 2 ) by setting 

51 = 711U1 + 712U2 (82) 

5 2 =72iU 2 + 722 Ui, (83) 



where 



1 _ n 2 (\ - 9-2R2\ 

711 = P \ 2 1 712 = P2- 2R ^ (84) 

1 — (r 



1 _ n 2 C\ - 9-2-Rn 

721 = P 4 1 722 = P T 2R * • (85) 

1 — p z 
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Note that 



< 7ii < 1 and < <y i2 < p, i G {1, 2}. 



(86) 



E.2 Expected Distortion 

To analyze the expected distortion we use a genie-aided argument. We first show that, 
under certain rate constraints, the asymptotic normalized distortion of the proposed 
scheme remains the same when a certain help from a genie is provided. To derive the 
achievable distortions it then suffices to analyze the genie-aided version. 

E.2.1 Genie-Aided Scheme 

In the genie-aided scheme, the genie's help is provided to the decoder. An illustration 
of this genie-aided decoder is given in Figure [9l The genie provides the decoder with 



U? 



us 



Ui 



u 9 



cG 
3 2 



Figure 9: Genie-aided decoder. 

the codeword pair (U*, U|). The decoder then estimates the source pair (Si, S2) based 
on (U|,U|) and ignores the guess (Ui,U2) produced in the first decoding step. The 
estimate of this genie-aided decoder is denoted by (Sf, SJf ), where 



s^ = tuu; + 712^ 

=72iU^ + 722 U|, 



(87) 



with 7x1, 712, 721, 722 as in (jMl) and (|85j) . Under certain rate constraints, the normal- 
ized asymptotic distortion of this genie-aided scheme is the same as for the originally 
proposed scheme. This is stated more precisely in the following proposition. 

Proposition E.l. For every 5 > and < e < 0.3 there exists an n'(5, e) > such 
that for all n > n'(5, e), 



Ie 

n 



IS1-S1I 



n 



Si - Sf || 2 j + 2a 2 (e + (44 v / TTe + 6l) 5) 
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whenever (Ri,R 2 ) is in the rate region 11(e) given by 



11(e) 



R2 < ^ lo g2 



1 (P 1 (l-p i )^N 



N(l - p 2 ) 

P 2 (l-p 2 ) + N 
N(l - p 2 ) 



K\e 



k 2 c 



1 T / Pi + P2 + 2pV7\P2 + N 



Ri + R 2 <^og 2y m _ p) 
where K\, k 2 and K3 depend only on Pi, P 2 , N, (1 and Q 2 , where 



K 3 e 



Ci 



Np 



Pi(i-p 2 ) + ivV p 2 

Proof. See Section IE. 31 
Corollary E.l. If (R\,R 2 ) satisfy 



— - and C2 



A(1~P 2 ) 
Pi(l-p 2 ) + iV' 



^ < 2 l0g2 V AT(l-p2) 

1 ( P 2 (l-p 2 ) + N 
R2< 2 l0g2 V AT(l-p2) 



Pi + P 2 < j log 2 ^ j^-^ 



then 



lim — E 

n— >oo 77, 



IS1-S1] 



< lim -E 

n— >oo 77, 



S1-S5: 



Gi|2 



5 -» 0. 



□ 



Proof. Follows from Proposition IE. II by first letting n — > 00 and then e — * and 



□ 



By Corollary IE.H to analyze the distortion achievable by our scheme it suffices to 
analyze the genie-aided scheme. This is done in Section IE.4I 



E.3 Proof of Proposition [ETT] 

The main step in the proof of Proposition IE . 1 1 is to show that for every (R\,R 2 ) 6 1Z(e) 
and sufficiently large n, the probability of a decoding error, and thus the probability 
of Si 7^ Sp, can be made very small. This step is done in the following section. The 
proof of Proposition IE. II is then completed in Section IE. 3. 21 



E.3.1 Upper Bound on Probability of a Decoding Error 

In this section we show that for every (Ri,R 2 ) € 11(c) and sufficiently large n, the 
probability of a decoding error, and thus the probability of Si / Sf, can be made 
very small. The hitch is that to upper bound the probability of a decoding error 
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for the proposed scheme, we cannot proceed by the method conventionally used for 
the multiple-access channel. The reason is that in the conventional analysis of the 
multiple-access channel it is assumed that the probability of the codewords Uj(j') does 
not depend on the realization of the codebook Cj. However, in our combined source- 
channel coding scheme, the probability of encoder i G {1,2} producing the channel 
input of index j G {1,2,... , 2 niJi } depends not only on the source sequence Sj, but 
also on the realization of Cj. Another reason the conventional analysis fails is that, 
conditional on the codebooks C\ and C 2 , the indices produces by the vector-quantizers 
are dependent. 

To address these difficulties, we proceed by a geometric approach. To this end, 
we introduce an error event related to a decoding error at the receiver. This event is 
denoted by £jj and consists of all tuples (si, S2, C%, C 2 , z) for which there exists a pair 
(ui,u 2 ) 7^ (u*,u|) in C\ x C 2 that satisfies Condition (f8Tj) of the reconstructor, and 
for which the Euclidean distance between ceitii + «2U2 and y is smaller or equal to 
the Euclidean distance between ct\\i\ + a 2 u.2 and y. More formally, £^ = £^ U £^ U 

£(Ui,U 2 ) where 

£- (ji = |(si,s 2 ,Ci,C 2 ,z) : 3ui G C\ \ {u*} s.t. \p- cos <(ui, u 2 )| < 7e, 

and ||y-(Qifii + a 2 u^)|| 2 < ||y-(aiu^ + a 2 u;)|| 2 | (89) 

£ u 2 = |( s i 1 s 2,Ci,C 2 ,z) : 3u 2 G C 2 \ {u^} s.t. \p - cos <(u*, u 2 )| < 7e, 

and ||y-(aiu* + a 2 ii2)|| 2 < ||y- (aiu* + a 2 u;)|| 2 l (90) 



^(Ui,U 2 ) = |(si,s 2 ,Ci,C 2 ,z) : 3ui € Ci \ {u{} and 3u 2 G C 2 \ {u* 2 } s.t. 

\p — cos<(ui,u 2 )| < 7e, 
and ||y - (aiiii + a 2 U2)|| 2 < ||y - («iui + a 2 U2)|| 2 J>, (91) 



andwherey = aiuj+aiuj+z. Note that a decoding error occurs only if (si, s 2 , C\, C 2 , z) G 
The main result of this section can now be stated as follows. 

Lemma E.l. For every 5 > and 0.3 > e > 0, there exists an n' 4 (5,e) G N such that 
for all n > e) 

< 115, whenever {Ri,R 2 ) G ft(e). 

To prove Lemma lE.H we introduce three auxiliary error events. The first auxiliary 
event £s corresponds to an atypical source output. More precisely, 



£s = < (si,s 2 ) £K" x 



1 II Il2 2 

— si — a 

n 



2 

> ea or 



^ II Il2 2 
— s 2 - a 

n 



>ea 2 



or |cos<(si,s 2 ) — p\ > ep >. (92) 
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The second auxiliary event is denoted by £ z an d corresponds to an atypical behavior 
of the additive noise: 



£z = S (si,s 2 ,Ci,C 2 ,z) : 



1„ M 

— z 

n 



> eN or -I (aiu?(si,Ci),z) I > JPiNe 
n 

or -| (a 2 u* 2 (s 2 ,C 2 ),z) \ > /fyVe). (93) 
n J 

Finally, the third auxiliary event is denoted by £x and corresponds to irregularities 
at the encoders. That is, the event that one of the codebooks contains no codeword 
satisfying Condition (|78p of the vector-quantizer, or that the two quantized sequences u* 
and u 2 have an "atypical" angle to each other. More formally, £x = ^Xi U£x 2 U£(Xi,x 2 ) 
where 

£xx = |(si,s 2 , d,C 2 ) : M G Ca s.t. ^1 - 2~ 2R i - cos <(si, m) < e^T^f^X 
£x 2 = |(si,s 2 ,Ci,C 2 ) : ^u 2 G C 2 s.t. jVl-2- 2 ^ - cos <(s 2 ,u 2 ) | < eyJl-2~ 2R ^ 



£(Xi,x 2 ) = \ (si,s 2 ,Ci,C 2 ) : \p- cos<(u*(si,Ci),u 2 (s 2 ,C 2 ))| >7e\. 



To prove Lemma lE.ll we now start with the decomposition 

Prfo] = Prfo n £| n ££ n £ z ] + Pr^l^s U 5x U f z ] Prfe U £ x U £ z ] 
*0 n n n £ z ] + Pr[£ s ] + Pr[£ x ] + Pr[fz] 



< Pr 

< Pr 



+ Pr 



£u 2 n^n£ x n£ z 



+ Pr 



^Ux.Ua) n £ s n f x n £ z + Pr[£ s ] + Pr[fx] + Pr[£ z ] 



(94) 



where we have used the shorthand notation Pr[£j,] for Pr[(Si, S 2 ,Ci,C 2 , Z) G and 
where denotes the complement of f^. Lemma |E. II now follows from upper bounding 
the probability terms on the RHS of (f94l) . 

Lemma E.2. For every 5 > and e > there exists an n^(<5, e) G N suc/i that for all 
n > n[ (5, e) 



Pr[£s] < 5- 

Proof. The proof follows by the weak law of large numbers. 



□ 



Lemma E.3. For every e > and 5 > i/iere exists an n' 3 (5,e) G N smc/i i/iaf for all 
n > n' 3 (S,e) 

Pr[£ z ] < 8. 

Proof. The proof follows by the weak law of large numbers and since for every e > 



sup 

ugR": 



Pr 



-| («iU,z) I > ^PiNe 
n 







as n —* 00, 



where i G {1, 2}. 



□ 
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Lemma E.4. For every 5 > and 0.3 > e > there exists an n' 2 (S,e) £ N such that 
for all n > n' 2 (5, e) 

Pr[£ x ] < 65. 

Proof. This result has nothing to do with the channel; it is a result from rate-distortion 
theory. A proof for our setting is given in Section IE.5.11 □ 

Lemma E.5. For every 5 > and every e > there exists some n'l(8,e) £ N such that 
for all n > n'l(5, e) 



Pr 



Pr 



<r a ■ fR/ 1 , ( Pi(l-~p 2 ) + N 



£u 2 n£§n5 x n5£ <$, ^/ J R 2 <-log 2 



1 /p 2 (l-^)+AT 



iV(l -p2) 



K 2 e 



Pr 



^(Oi,u 2 ) nf s nf xnfS <S, if R l + R 2 < -\og 2 



1, ( Pi + P 2 + 2p^/7\P 2 ~ + N 



2"^\ N{l-p 2 ) 
where k\, k 2 , and K3 are positive constants determined by Pi, P 2 , and N . 
The proof of Lemma IE.5I requires some preliminaries. To this end, define 
w(si,s 2 ,Ci,C 2 ,z) = Ci(y - a 2 u 2 ) + ( 2 a 2 u 2 , 

where 



K 3 e 



Ci 



Np 



- 

P 1 {l-p 2 ) + N\jp 2 



and C2 



Pi(l-p 



r,2 1 



Pi(l-p 2 ) + iV' 



(98) 



(99) 



In the remainder we shall use the shorthand notation w instead of w(si, s 2 ,Ci,C 2 , z). 
We now start with a lemma that will be used to prove (f95|) . 



Lemma E.6. Let ipj £ [0, tt] be the angle between w and ui(j), and let the set be 
defined as 



ci A 



(si,s 2 ,Ci,C 2 ,z) :3u x (i) GCi\{uj;} s.t 



PU1- p 2 ) + Np 2 „ , . 

cos ^V M(i-^) + iv -•')■ (100) 

where k" is a positive constant determined by P\, P 2 , N, (j and C, 2 . Then, 
and, in particular 



Pr 



< Pr 
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Proof. We first recall that for the event £^ to occur, there must exist a codeword 
ui(j) G C\ \ {u^} that satisfies 

|p-cos<(u 1 (i),u*)| < 7e, (101) 

and 

||y - (aiui(i) + «2U^)|| 2 < ||y - (aiu? + a 2 u^)|| 2 . (102) 

The proof is now based on a sequence of statements related to Condition (jlOip and 
Condition (fTMD : 

A) For every (si, s 2 ,C\,C 2 , z) G £^ and every u G Si, where Si is the surface area 
of the codeword sphere of C\ defined in the code construction, 



\p- cos<(u,U2)| < 7e => np\fP^P2- (aiVL,a 2 v? 2 ) < 7n^P 1 P 2 e. (103) 

Statement A) follows by rewriting cos<(u,U2) as (u, u^) /(||u|| ||u2 1|), and then 
multiplying the inequality on the LHS of (|103p by ||aiu|| • Ha^u^H and recalling that 
||aiu|| = y/nPi and that 1 1 «2 U2 1 1 = V n P2- 

B) For every (si, S2, C±, C 2 , z) G £^ D <? z and every u G Si 
||y - (aiu + «2U2)|| 2 < ||y - (aiuj + a 2 U2)|| 2 



=> (y - a 2 u^, am) > nPj - ny/P^e. (104) 

Statement B) follows from rewriting the inequality on the LHS of (|104p as ||(y — 
Q2U2) — aiUi(j)|| 2 < || (y — c^u^) — Qiu^|| 2 or equivalently as 

(y - a 2 U2, aiui (j)) > (y - a 2 u* 2 , aiu*) 
= (aiu^ + z,aiu*) 

= ||aiuj|| 2 + (z,aiu*) . (105) 

It now follows from the equivalence of the first inequality in (|104p with ()105p that 
for (si, S2, Ci, C2, z) G £ z , the first inequality in (|104p can only hold if 



(y - a 2 u^, aiu) > nPi - nVPiVe, (106) 

thus establishing B). 

C) For every (si, S2, Ci, C 2 , z) G n £ z and every u G Si, 

(|p - cos<(u,U2)| < 7e and ||y - (qiu + a 2 U2)|| 2 < ||y - (aiu* + a^u^H 2 ) 



(107) 



aiu 



w 



< nPi - 2 (Cm (Pi - TPiVe) + (2ny / PiP 2 (p - 7e)) + ||w|| 2 ^ . 
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Statement C) is obtained as follows: 

||aiu — w|| 2 = ||aiu|| 2 — 2 (aiu, w) + ||w|| 2 



|aiu|| 2 - 2 (Ci (aiu, y - a^u^) + C2 («iu, a^UjJ)) + ||w| 



a) 



< nPi - 2 \^in [Pi - yJPjle} + C, 2 nJp^P 2 (p - 7e) J + 
where in a) we have used Statement A) and Statement B). 

D) For every (si , S2 , C\, C 2 , z) £ £^H £^ 



Iwll 2 , 



||w|| 2 < n [CfPi + 2C1C2 VP^PiP + (I (Pi +N) + Kej , (108) 
where k depends on Pi, P 2 , N, (1 and C2 only. 

Statement D) is obtained as follows 

ll w l| 2 = Cl l|«2U2 || 2 + 2ClC2 («2U2,y - a 2 U 2 ) + Cf ||y - 02U2|| 2 

= (fnP 2 + 2C1C2 ((a 2 u* 2 , ai ul) - (a 2 u* 2 , z» + C 2 2 (||aiW|| 2 + 2 («iu*,z) + ||z|| 2 ) 

< CfnPi + 2C1C2 (ny/P^p + 7e) + n^/P^Ne) + Cl (nPi + 2n^/P\Ne + nN(l + e ; 



< n (Ci 2 Pi + 2CiC2\/^/5 + C 2 2 (^1 + N) + ne) , 
where in a) we have used that (si ,s 2 ,Ci,C 2 ,z) £ £%. 

E) For every (si, s 2 ,Ci,C 2 ,z) S £ \ n £^ an d an arbitrary u e Si, 

cos<(u,U2)| < 7e and ||y - (a x vi + a 2 U2)|| 2 < ||y - (qhu* + a 2 u* 2 )\\ 2 ) 

=> (109) 

|qiu — w|| 2 < T(e)^ , 
where 

PiiV(l-p 2 ) , , 
T{e) = n p 1 (l-pi) + N +nKe > 

and where «/ only depends on P, N\, N 2 , £1 and C, 2 . 

Statement E) follows from combining Statement C) with Statement D) and the 
explicit values of £1 and Q 2 given in (|99j) . 
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F) For every u G «Si, denote by <p G [0, it] the angle between u and w, and let 



5(8!, 82,^,113^) = < u G 5] : cosv? > W ^7f3^2y^v ~ I ' 

where k" only depends on P, N\, N2, Ci an d C2, and where we assume e sufficiently 
small such that 



Pi(l- p 2 )+Np 2 



II ^ rv 

k e > 0. 



Then, for every (si, S2, Ci, C2, z) G £^r\£^, 
(|p - cos<(u,U2)| < 7e and ||y - (qiu + a 2 U2)|| 2 < ||y - (aiu* + c^u^H 2 ) 



u G B(si,s 2 ,Ui,U2,z). 



(110) 



Statement F) follows from Statement E) by noting that if w 7^ and 1— T(e) / {nP\) > 0, 
then 



|aiu|| = 
laju — w 



nPi 

II 2 < T(e) 



cos <(u, w) > \ 1 



m 

nPi 



To see this, first note that for every a\u, where u G <Si, satisfying the condition on 
the LHS of (jllOp lies within a sphere of radius y / i(e) centered at w. And for every 
u G Si we have that a\u also lies on the centered M n -sphere of radius \pnP\. Hence, 
every u G satisfying the condition on the LHS of (jllOp lies in the intersection of 
these two regions, which is a polar cap on the centered sphere of radius \fnP\. An 
illustration of such a polar cap is given in Figure [TU1 The area of this polar cap is 



WnP\ 











Figure 10: Polar cap of half angle ip on an R n -sphere of radius \fnP\. 

outer bounded as follows. Let r be an arbitrary point on the boundary of this polar 
cap. The half- angle of the polar cap would be maximized if w and r — w would lie 
perpendicular to each other, as is illustrated in Subplot b) of Figure [TT1 Hence, every 
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w 





aj b) c) 

Figure 11: Half- angle of cap for different constellations of w and r. 



u G satisfying the upper conditions of (|11U|) also satisfies 



cos ip > \ / 1 



T(e) 
nPi 



,i2 



P l {l-p)+Np 
V Pi(l-p 2 )+iV " c ' 

where we assume e sufficiently small such that 1— T(e)/(nPi) > and where «/' = k'/P\. 



The proof of Lemma IE.6I is now concluded by noticing that the set £'^ , defined in 



(jlOOp . is the set of tuples (si, S2, C\, C 2 , z) for which there exists a ui(j) G C\ \ {u^} 
such that ui(j) G B(si, s 2 , uf, u|, z). Thus, by Statement F) and by the definition of 



£{j in fl89]) it follows that 



and therefore 



Pr 



< Pr 



u 



We now state one more lemma that will be used for the proof of (I95p . 
Lemma E.7. For every A G (0, 1], let the set Q be given by 

g = {(si,s 2 ,Ci,C 2 ,z) : 3ui(j) G C x \ {u^} s.t. cos <(w, VL\{j)) > A} , 
where w is defined in (J98]). Then, 

Ri < -^log 2 (l- A 2 )] (hm Pr[0|££j =0, e>0 



□ 



(111) 



Proof. The proof follows from upper bounding in every point on S± the density of every 
u i(j) S Ci \ {u^} and then using a standard argument from sphere-packing. The proof 
is given in Section [E. 5. 21 □ 
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We next state two lemmas for the proof of (197)) . These lemmas are similar to 
Lemma lE.61 and Lemma lE.71 

Lemma E.8. For every sufficiently small e > 0, define the set £'~ - as 

(Ui,U 2 ) 

< S ( / (ji t)2) ^{(s 1 ,s 2 ,C 1 ,C 2 ,z):3u 1 (i)GCi\{ul} and 3u 2 (£) E C 2 \ {u* 2 } s.t. 

cos<(ui(j),u 2 (f)) > p- 7e and cos <(y, axM\{j) + a 2 u 2 (f)) > A(e)}, 

where 



A(e) 



1 Pi + P 2 + 2p^/7\P 2 - g'e 
Pi + P 2 + 2p^PTP 2 + N + £ 2 e 



and where £' and £ 2 depend only on P\, P 2 and N . Then, for every sufficiently small 
e > 



and, in particular 



Pr 



^(Ui,u 2 ) n <Sz n <Sg n <Sx 



< Pr 



Proof. We first recall that for the event ^ s to occur, there must exist codewords 
ui(j') eCi \ {u|} and u 2 (£) G C 2 \ {ug} such'that 



|p-cos<(ui(j),u 2 (f))| < 7e, 



and 



|y - (ariui(j) + a 2 u 2 



< lly - («i u i + «2U 



*M|2 



(112) 



(113) 



The proof is now based on a sequence of statements related to Condition (|112p and 
Condition ([TTHjh 

A) For every (si , s 2 , d , C 2 , z) G n , 



|y - (aiui(j) + a 2 u 2 



2 < ||y- (aiuj + a 2 u^|| 2 



(114) 



(y, a x ui(j) + a 2 u 2 (£)) > n (p x + P 2 + 2~pJl\P 2 - fre) , 
where £i only depends on Pi, P 2 and iV. 
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Statement A) follows by rewriting the LHS of (|114p as 

2(y,aiui(j) +a 2 u 2 (£)) > 2(y,a 1 u* 1 + a 2 ul) + [|aiUi(j) + a 2 u 2 (t)\\ 2 - ||aiUj + a 2 u 2 || 2 

= ||aiUj + a 2 U2|| 2 + 2 (z, a<iu* + a 2 U2) + ||aiui(j) + a 2 u 2 ( 

a) 



>2n(P 1 + 2p v / PiP 2 (l - 7e) + P 2 + + ^ 



2n(Pi + P 2 + 2pv / ^-6e), (H5) 



where in a) we have used that (si, s 2 , C±, C 2 , z) G ^ fl ^ an d that 
Wmu^j) + a 2 u 2 (£)\\ 2 >0. 

B) For every (si, s 2 , Ci, C 2 , z) G ^x n ^z> 



||y|| 2 < n [P x + 2p^/pP 2 + P 2 + N + £ 2 eJ , 
where £ 2 only depends on Pi, P 2 and N. 
Statement B) is obtained as follows: 

||y|| 2 = ||aiu^|| 2 + 2{aiM\,a 2 M* 2 ) + ||a 2 u^|| 2 + 2 ((amj, z) + (a 2 u^,z)) + ||z|| 2 
< nPi + 2npy / P 1 P 2 (l + 7e) + nP 2 + 2n^P x Ne + 2ny / P 2 Ne + niV(l + e) 



< n (Pi + 2pyJP]P 2 + P 2 + N + £ 2 ej 
where in a) we have used that (si, s 2 , C\, C 2 , z) G n 

C) For every (si , s 2 , d , C 2 , z) , 



u lJ u 2 



(116) 



aiUi(j)+Q2U 2 (^)|| 2 <nfp 1 + 2p v / A^ + ^2+6e) • 
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Statement C) follows by 

||aiui(j) + a 2 u 2 (^)|| 2 = ||aiui(j)|| 2 + 2 (aiUi(j), a 2 u 2 (i)} + ||a 2 u 2 

a) . 

< nP 1 + 2n~pyJP l P 2 (l + 7e) + nP 2 



= n (Pi + 2p^/I\P 2 + P 2 + £ 3 e) . 

where in a) we have used that multiplying the inequality on the LHS of (|116p by 
||aiui(j)|| • ||a 2 u 2 (£)|| and recalling that [|aiui(j)[| < \fnP\ and that ||a 2 u 2 (£)|| < 
^JnP 2 gives 

i \J P\P 2 p - (aiui(j),a 2 u 2 (^)) I < 7ny / P 1 P 2 e, 



n \ 



and thus 



(aiUi(i),a 2 U2^)) <nyJP 1 P 2 p{l + 7e), 

thus establishing C). 

D) For every (si, s 2 , d, C 2 , z) G n ££, 

(|p-cos<(u 1 (i),u 2 (^))| < 7e 

and ||y - (qiui(j') + a 2 u 2 (f))|| 2 < ||y - (aiuj + a 2 U2)|| 2 ) 

=4> cos<(y,aiUi(j) + a 2 u 2 (f)) > A(e). 

Statement D) follows by rewriting cos<(y, aiUi(j') +a 2 u 2 (^)) as 

(y,aiui(j) + a 2 u 2 (£)) 



cos<(y,aiUi(j) +a 2 u 2 (£)) 



|y|| • ||aiui(j) + a 2 u 2 



and then lower bounding (y, aiUi(j) + a 2 u 2 (£)) using A) and upper bounding ||y|| 
and ||aiUi(j) + a 2 u 2 (^)|| using B) and C) respectively. This, yields that for every 

( Sl ,s 2 ,Ci,c 2 ,z) ee^nez, 

cos<(y,aiUi(j) + a 2 u 2 (f)) 



> Pl + p 2 + 2 p^p[p^-^e 



y/Px +P 2 + 2p^P 1 P 2 +N + ~&eyfp\ +P 2 + 2p^P 1 P 2 + £ 3 e 



> 



; Pi + P 2 + 2~pJf\P 2 - ge 
Pi+P 2 + 2~pJPTP 2 + N + &e 



A(e) 
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Lemma lE.81 now follows by D) which gives 
and therefore 

Pr[£ (t ^ 2) |^ n fi n £fc] < Pr[f^ i)Ca) |^ n Eg n . □ 

We now state the second lemma needed for the proof of (|97j) . 
Lemma E.9. For every G (0, 1] and A G (0, 1], let the set Q be given by 

G = {(si,s 2 ,Ci,C 2 ,z) : 3ui(i) G Ci \ {uj} ,u 2 (£) G C 2 \ {u* 2 } s.t. 

cos<(ui(j),u 2 (£)) > 0,cos<(y,aiUi(j) + a 2 u 2 (f)) > A}. 



Then, 



(Pi + P 2 <-ilog 2 ((l-0 2 )(l-A 2 ))) 

lim Pr[a|%n^ a ] =0, e>o). 



(117) 



Proof. The proof follows from upper bounding in every point on 5j, i G {1,2}, the 
density of every Uj(j) G Cj \ {u*} and then using a standard argument from sphere- 
packing. The proof is given in Section IE.5.31 □ 



Proof of Lemma IF, 51 We first prove ([95 



Pr 



a) 

< Pr 

6) 

< Pr 



4 n^nf|n^ 



£' 



Ui 



(118) 



where a) follows by Lemma lE.61 and 6) follows because C <Sx 1 - The proof of (|95p is 
now completed by combining (|118[) with Lemma lE.71 This gives that for every 5 > 
and every e > there exists some n' 41 (S,e) such that for all n > n' 41 (5,e) we have 



Pr 



< 5 whenever 



i?i < -- log 2 



< g lo S2 



iV(l-p 2 ) 

+ k e 



Pi(l-p 2 ) + iV 

Pi(l-p 2 )+iV 
iY(l-p 2 ) 



where k\ is a positive constant determined by Pi, P 2 , N, £i and £ 2 . A similar argument 
establishes PU|) . 

We turn to the proof of ([97]) . 



Pr 



^(Ui,u 2 ) n n n 



a) 

< Pr 

6) 

< Pr 



£ (Ui,u 2 )l f Xi n£ x 2 



(119) 
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where a) follows by Lemma IE. 81 and b) follows because C n £ x ) . The proof 
of (|97p is now completed by combining (|119p with Lemma IE. 91 which gives that for 
every 5 > and every e > there exists some n' i3 (5, e) such that for all n > (5, e) 
we have Pr 



^(Ui,u 2 ) n n n 



Ri + R2<^ log 2 



< 5 whenever 

Pi + P 2 + 2~pj¥\P 2 + N + & 
(N+(e + ^)e)(l-p 2 + ^e) 



1, /Pi + P 2 + 2 / o v / 7Vf^ + A r 

^2 log2 V ivO^) K3e 

where K3 is is a positive constant determined by Pi, P 2 and iV. 
The proof of Lemma IE. II now follows straight forwardly. 



□ 



Proof of Lemma \E.ll Combining (|94p with Lemma IE.21 Lemma IE. 31 Lemma IE.4I and 
Lemma lE.51 yields that for every 5 > and 0.3 > e > there exists some n 4 (<5, e) £ N 
such that for all n > n 4 (5, e) 



Prfoj] < 115 



if (RuBa) € 11(e). 



□ 



E.3.2 Concluding the Proof of Proposition IE. II 

We start with four lemmas. The first lemma upper bounds the impact of atypical 
source outputs on the expected distortion. 



Lemma E.10. For every e > 



Proof. 



Ie 

n 



|Si| 



1 



|Si| 



Pr[fs 



Pr[«Ss] <<J 2 (e+ Pr[f s ]). 



-E ||Si| 
n 



Ie 

n 



|Si| 



Pr[f. 



<a 2 -<x 2 (l-e)Pr[£|] 

= <7 2 -<7 2 (l-e)(l- p r[£g]) 

= a 2 e + C T 2 (l-e)Pr[ < Ss] 
<a 2 (e+ Pr[f s ]). 



□ 



The second lemma gives upper bounds on norms related to the reconstructions §1 



and if. 



Lemma E.ll. Let the reconstructions s\ and sf 6e as defined in ()82p and (j87j) . Then, 



Isill 2 < 4ncr 2 



I if || 2 < 4n<r 2 



|ii - si|| 2 < I6na^ 
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Proof. We start by upper bounding the squared norm of §1 

||si|| 2 = H711U1 + 712U2II 2 

= Tiilluill 2 + 2711712 (ui,u 2 ) + T12 1 1 u 2 1 1 2 

< 7nl|ui|| 2 + 27ii7i 2 ||ui||||u 2 || + 7 2 2 ||u 2 || 2 

= (7n||ui|| + 7i2||u 2 ||) 2 
a) 

< na 2 {l + p) 2 

< Ana 2 , 

where in a) we have used (f86l) . i.e., that 711 < 1 and 7 l2 < p, and that ||uj|| < V na 2 , 
i G {1,2}. The upper bound on the squared norm of s^p is obtained similarly. Its proof 
is therefore omitted. The upper bound on the squared norm of the difference between 
si and now follows easily: 



sf -si|| 2 < ||s^|| 2 + 2||s^||||si|| + ||si 



sGi|2 



,Gi 



+ llsil 



< IQna 2 . 



□ 



The next two lemmas are used directly in the upcoming proof of Proposition IE. 11 
They rely on Lemma IE. 101 and Lemma IE. Ill 



Lemma E.12. 

1 



Proof. 



n 



Si,Sf - Si VI < a 2 (e + 17Pr[£ s ] + 4^/TT7 Pr ) • 



n 



Si, Sf — Si 



n 
+ 



Si, s x — s 



Pv[£, 



1 



n 
n . 



Si, sf 



u 



Pr[££n£ c ] 



Si, Si 



G 



a) I 

< — E 

n 

+ 



1 



n 



Si|| 2 + ||Sf -Sill 2 
SilHIS? -S 



Pr[£ S ] 



n 



|Si| 



Pr[5 s ] + 16<7 2 Pr[£ s ] 



+ y / a 2 {l + e)Vl6a 2 Pi[S 



'UJ 



c) 



< a 2 {e+ Pr[£ s ]) + 16cj 2 Pr[ < E: s ] 
+ 4a 2 ^TT7Pr[f i:j ] 

< a 2 (e + 17Pr[£ s ] + 4VT+ePr[£ 6 ] ) . 



(120) 
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In the first equality the third expectation term equals zero because by £^ we have 
||sp — si || = and by <?g the norm ||si || is bounded. In a) we have used two inequalities: 
in the first term, the inner product is upper bounded by using that for any two vectors 
v € W 1 and w G K n 



(v, w) | < ||v|| • ||w|| 

<^(l|v|| 2 + ||w|| 2 ) 
< ||v|| 2 + II wll 2 . 



(121) 



The second term is upper bounded by the Cauchy-Schwarz inequality and by Pr [£g n £ jv] < 
Pr [Sfj] . In b) we have used Lemma IE. 1 1 1 and in c) we have used Lemma IE. 101 □ 



Lemma E.13. 



Proof. 



1 

-E 

n 



n 



|Si|| 2 - ||Sf || 2 1 < 8a 2 Pr[£ d 



|S?|| 2 



1 

= -E 

n 

+ i E 

n 

a) I 

< -E 

n 



|Sf f\£- 



u 

G||2|cc 



sxir — Iis5r Irl^-o 

Si|| 2 + ||Sf|| 2 |£ 



Prfo 
Pr 

Prfo 



£ - 



< 8cr 2 Pr[%] 



where a) follows since conditional on£? we have si = sj* and therefore — 
0, and where b) follows by Lemma IE. Ill 



£G||2 
5> 1 



□ 



Proof of Proposition \E.li We show that the asymptotic normalized distortion resulting 
from the proposed vector-quantizer scheme is the same as the asymptotic normalized 
distortion resulting from the genie-aided version of this scheme. 



Ie 

n 



ISi-Sil 



Ie 

n 



I Si - s^ 



G||2 



2 Ie 

n 



E[||S 1 || 2 ]-2E 
-EfllSxH 2 ] +2E 

Si, sf — s 



Si, Si 



+ E 



I Si || 2 



Si.S? 



n 



|S?|| 2 



|Si|| 2 - ||Sf || 2 



a) 



< 2a 2 (e + 17Pr[£ s ] + 4 V / TT^ Pr [£$] ) 
+ 8a 2 Pr[f c ] 

2a 2 (e + 17Pr[£: s ] +4(^/TT7 + l) Pr 



(122) 
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where in step a) we have used Lemma IE. 121 and Lemma IE. 131 Combining (|122l) with 
Lemma lE.21 and Lemma |E. II gives that for every 5 > and 0.3 > e > 0, there exists an 
n'(5,e) > such that for all (Ri,R 2 ) £ 11(e) and n > n'(8,e) 



n 



ISi-Sil 



ii 



|Si 



jGi|2 



< 2a 2 (e + (AAVi + 1 + 61) S) 



□ 



E.4 Upper Bound on Expected Distortion 

We now derive an upper bound on the achievable distortion for the proposed vector- 
quantizer scheme. By Corollary IE.H it suffices to analyze the genie-aided scheme. 
Using that Sf = 711U* + 712U2, we have 



n 



I Si - 



Gi|2 



2 7 iiE[(Si,Uj)]-2 7 i2E[<Si,U2)] 



+7i 2 iE[||U^|| 2 ] 



27ii7i 2 E[(Ut,U^)]+7i 2 2 E ||U; 



* 1 1 2"1 



o 2 - 27iiiE[(Si,UJ>] - 2 7i2 Ie[(S 1 ,U*)] 

n n 

1. 



+7fi^(l - 2-^) + 27 ll7 i2-E[(U 1 ,U^)] 
+7 12 o- (1-2 2 ), 



(123) 



where in the last equality all expected squared norms have been replaced by their 



explicit values, i.e., E ||Si| 



no and Ell U,; 



no 



(1 - 2~ 2R >) for i G {1,2}. 



The remaining expectations of the inner products are bounded in the following three 
lemmas. 

Lemma E.14. For every 5 > and 0.3 > e > and every positive integer n 

^E[(Si,Uj>] >(J 2 (l-2- 2Rl )(l-2e)(l-75). (124) 

Proof. 



n 



n 



-E[<Si,Ui>] 



n 



|Si||||U*||cos<(Si,U*) 



>o 



n 



|Si||||Ut||cos<(Si,U*) £§n£% ■ Pr[£|n££] 
> - ^no 2 (l - e)Jno 2 (l - 2- 2R ^)Vl - 2~ 2 ^(1 - e) Pr[£| D ££] 



> _ 2 -2fli )(1 _ 2e) (1 _ p r[fs ] _ Pr[< r x]) ; 

where in the first equality the first expectation term is non-negative because conditioned 
on £ x either UJ = or, if UJ / 0, then cos (<(Si, UJ)) > 0. 

By Lemma lE.21 and Lemma lE.4l it now follows that for every 5 > and 0.3 > e > 
there exists an n'(8, e) E N such that for all n > n'(5, e) 



iE[(Si,UI)] > o 2 (l - 2- 2Rl )(l - 2e)(l - 75). 



□ 
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Lemma E.15. For every 5 > and 0.3 > e > 0, there exists an n' 2 {5, e) G N swc/i t/iai 
/or a// n > n' 2 ((5, e) 



Proof. 



-E[(UI,U5)1 < <r 2 6<5 + (7 2 p(l - 2- 2fil )(l - 2- 2i?2 )(l + 7e). 
n 

iE[(U?,U3>] = iE[(Ul,U3>|£ x ] Pr[£x] + ~E[(U?,U|)|£ X ] Pr[£ 
< -E[||U^||||U5[||f x ] Pr[fx] + -E[(Ui,U5) \S 



•x] 



< a 2 -^/(l - 2" 2 «i)(l - 2- 2 *2) Pr[£ x ] 



n 



p(l + 7e)^na 2 (l - 2- 2R ^)^na 2 (l - 2^) 



< a 2 Pr[£ x ] + a 2 p(l - 2- 2/?1 )(l - 2- 2R2 )(l + 7e). 

Thus, it follows by Lemma IE, 41 that for every 5 > and 0.3 > e > there exists an 
n' 2 (S, e) G N such that for all n > n' 2 (S, e) 



E[<UJ,U5)] < cj 2 65 + a 2 p{l - 2~ 2Rl )(l - 2- 2R2 ){l + 7e). 



□ 



(125) 



Lemma E.16. For every 6 > and 0.3 > e > 0, there exists an n'(5, e) G N suc/j i/iai 
/or a// n > n'(5, e) 

-E[(Si, U5>] > o- 2 p(l - 2- 2i?2 )(l - e) 3 - a 2 (e + 215 + 6<5e) . 
Proof. We begin with the following decomposition: 

-E[(S 1 ,XJl)] = -E[{8i 1 X^)\S a ue x ,] Pr[£ s U£x 2 ] 

n n 1 1 J 

+ ^e[(Si,U5> l^n^ij Pr[^n£ x J. 

The first term on the RHS of (|125p is lower bounded as follows. 

-E[(S 1; U*) \£sU£ X2 ] Pv[£ s u£x 2 ] 
n 

> --E[||Si|| 2 + ra| 2 |fsUf Xa ] Pr[£ S U£ X2 

n 

> -i(E[HSi[| 2 |fs] Pr[fs] 

+ E[||s 1 || 2 |^n£:x 2 ] Pr^n-Sx] 

+ ||U^|| 2 (Pr[£ s ] + Pr[«S x ]) N 



> - (o- 2 (e + Pr[£ s ]) + ^ 2 (1 + e) Pr[fx] 

+ a 2 (l-2- 2 ^)(Pr[f s ] + Pr[f x ]) 

> -o- 2 (e + 2 Pr[5 s ] + (2 + e) Pr[5 x ] ) , 



(126) 
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where in o) we have used (|121fl . in b) we have used that £ x =2 £x 2 i an d i n c ) we have 
used Lemma |E. 101 

We now turn to lower bounding the second term on the RHS of (I125D . The proba- 
bility term is lower bounded as follows 



Pr[££n£ X2 ] =1- Pr[£ s U£ X2 ] 

>l-(Pr[f s ]+ Pr[£x])- 



(127) 



To lower bound the expectation term, we represent u* as a scaled version of s, corrupted 
by an additive "quantization noise" Vj. More precisely, 



u, 



where 



uj 



• COS <(Sj, Uj 



»6{1,2}. (128) 



With this choice of z/j, the vector Vj is always orthogonal to Sj. By (|128p . the inner 
product (SijU^) can now be rewritten as z^ 2 (Si, S2) + (Si, V2). Hence, 



(Si,u^) £|n£ X: 

- b Si,S 2 



i*(si,s 2 > (Si,S 2 ) = (s 1 ,s 2 ),finfx 2 



+ E» 1)¥a (si,V 2 ) (Si,S 2 ) = (si,s 2 ),^nfi 2 



-Si,S 2 



6) 

> E Si,S 2 



iu:i 



(Si,S 2 )E %X2 [cos<(s 2 ,U^)|(Si,S 2 ) = (si,s 2 ),^n«E- X2 



|U2||||Si|| cos (<(Si, S 2 )) Vl-2- 2 «2(i _ e > 



£ s n £ X2 



> ^na 2 {l - 2- 2R 2)^/na 2 {l-e)p{l - e)Vl-2- 2R *(l - e) 

> npa 2 {\ -2~ 2R2 )(l-ef, 



(129) 



where we have denoted by ^ the random codebook of user i S {1,2}, and where in a) 
the second expectation term is zero because for every (si,s 2 ) £ £g 



E % (si,V 2 ) (Si,S 2 ) = (s 1)S2 ),f X2 



0. 



This holds since in the expectation over the codebooks ^2 with conditioning on £ x , 
for every v 2 6 R n the sequences v 2 and — v 2 are equiprobable and thus their inner 
products with si cancel off each other. Inequality b) follows from lower bounding 
cos<(s 2 ,U2) conditioned on £ x combined with the fact that conditioned on £g the 
term cos<(Si,S 2 ) is positive. Inequality c) follows from lower bounding ||Si|| and 
cos<(Si,S 2 ) conditioned on 
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Combining (fT25l) with (fT26jh (fT27j) and (fT29|) gives 

iE[(Si,U5)] > -a 2 (e + 2Pr[£ s ] + (2 + e)Pr[£ x ]) 

n 

+ a 2 p(l - 2- 2/?2 )(l - e) 3 (1 - (Pr[5 s ] + Pr[5 x ])) 
> a 2 p(l - 2- 2fl2 )(l - e) 3 - a 2 (e + 3Pr[5 s ] + (3 + e) Pr[5 x ]) • 



Thus, by Lemma IE.2I and Lemma IE.4I it follows that for every 5 > and 0.3 > e > 
there exists an n'(6, e) 6 N such that for all n > n'(5, e) 



1 



-E[(Si,U5)] > a 2 p(l - 2~ 2i?2 )(l - e) 3 - a 2 (e + 215 + 6<5e) 



□ 



The distortion of the genie-aided scheme is now upper bounded as follows 



1 



n 



Sf|| 2 



a 2 - 2tu-E[(Si,U}>] - 2 7 i2-E[(S 1 ,U5)] 
n n 

+ 7l > 2 (l - 2~ 2R i) + 2 7 ii7i2^E[(Ut, V* 2 )[ 
+ 7l V 2 (l-2- 2 ^) 



a) 

< a z 2 



where in a) we have used Lemma IE. 141 Lemma IE. 151 and Lemma IE. 161 and where 

lim m,e) = 0. 

E.5 Proofs of Lemma IE.4L Lemma IE. 71 and Lemma IE. 91 

The proofs in this section rely on bounds from the geometry of sphere packing. To this 
end, we denote by C n (ip) the surface area of a polar cap of half angle tp on an M n -sphere 
of unit radius. An illustration of C n (tp) is given in Figure [T2l Upper and lower bounds 




Figure 12: Polar cap of half angle (p. 

on the surface area C n (tp) are given in the following lemma. 
Lemma E.17. For any <p S [0, 7r/2], 



r (S + 1) sin^- 1 ) 



nT (2±i) ^cosv? 



1 



■ tan tp) < 

II J C n (7Tj 



C n (f) < r(f + l)sin("- 1 ) 



rar(^±±)0Fcos<^ 
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Proof. See [231 Inequality (27)]. □ 

The ratio of the two gamma functions that appears in the upper bound and the 
lower bound of Lemma lE.171 has the following asymptotic series. 



Lemma E.18. 



T(x + l) 1 1 5 21 



X11- — + + t— Q - — — , + 



Proof. We first note that 



v V 8x 128x2 1024x 3 32768x 4 

and in particular 

lim _ i — !_2Z = i. 

x— t-oo r(x)y / X 

r(x + i) (2x-l)l! 

2*(a;-l)! 

=l( 2 ;)v7, (130) 

where £!! denotes the double factorial of £. The proof now follows by combining (|130p 
with 

2i\ f / 1 1 5 21 

1 — ~ h —7-^7 + — o — — j + ■ ■ ■ 



7T 



x / y 7 ^ V 8^ 128x 2 1024x 3 32768x 4 
which is given in [24|, Problem 9.60, p. 495]. □ 

Before starting with the proofs of this section, we give one more lemma. To this end, 
whenever the vector-quantizer of Encoder 1 does not produce the all-zero sequence, de- 
note by Si(si,Ci) the index of u* in its codebook C\. And whenever the vector-quantizer 
of Encoder 1 produces the all-zero sequence, let ?i(si,Ci) = 0. Further, let Ai(-) denote 
the measure on the codeword sphere S\ induced by the uniform distribution, and let 
/ Al (-) denote the density on S\ with respect to Ai(-). Similarly, for Encoder 2 define 
?2 (s 2 ,C 2 ) and/ A2 (-). 

Lemma E.19. Conditional on ?i(Si,^fi) = 1, the density o/Ui(j) is upper bounded 
for every j £ {2, 3, . . . , 2 nRl } and at every point u £ S\ by twice the uniform density: 

/ Al (LMj) = u| fl (Si,^i) = 1) < 2 • * u G Si, j € {2,3, ... , 2^}. 

And similarly for Encoder 2. 

Proof. We first write the conditional density as an average over cos <(Si, Ui(l)). Since 
conditioned on ?i (Si, ^i) = 1 we have cos <(Si, Ui(l)) € [y/l - 2~ 2H ^ (1-e), y/1 - 2~ 2 «i (1+ 
e)], we obtain 

/ Al (Ui(j)=u| ? i(Si,^) = l) 

p i-^l-2- 2R l(l+e) 

= I / Al (Ui(j)=u|Si=si, fl (si,^ 1 ) = l,cos<(si,Ui(l))=a) 

JsiSM™ J y 1— 2~ 2 - R i (1— e) 

/(Si = si,cob<(si,Ui(1)) = aki(Si,«i) = l)dodsi. (131) 
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The proof now follows by upper bounding the conditional density 

f Xl (Ui(i) =u|Si =si, a (si,«i) = 1,cob<(si,Ui(1)) =a) 
To this end, define for every a G [Vl - 2- 2R ^(l - e), \f\ - 2" 2i? i(l + e)] 



£ (si) = {u € Si : cos<(si,u) - \/l - 2" 2fll < a-y/l- 2_ 2jRi ] } » 



and 



' a c {si) = {u6 5i: cos<(si,u) - - 2~ 2R i > a-y/l- 2~ 2R i } . 



The conditional density can now be upper bounded by distinguishing between u G 
^ a (si) and u G i^(si). If u G ^ a (si), then the conditional density is zero because the 
fact that ?i(si,^i) is 1 implies that 



cob<(si,Ui(j)) - \A - 2- 2/? i 



> 



^1 - 2- 2 «i 



VjG {2,3,..., 2 



}• 



And if u G i^(si) the conditional density is uniform over @£(si), i.e. 
f Xl (ViU) = u l s i = si,a(si,*i) = l,cos<(si,U 1 (l)) = a) = u, 



uG^(si), 



for some w > 0. Thus, 

/ Al (Ui(i) = u|Si = 81^1(81,^1) = l,cos<(si,Ui(l)) = a) < v, 



(132) 



VuGSi,Si GM n ,aG [\Tl - 2" 2 ^ (1 - e), y/l - 2" 2 ^ (1 + e)]. 

It now remains to upper bound v. To this end, notice that the surface area of @ a (s{) 
never exceeds half the surface area of Si . This follows since Vl - 2- 2R i(l-e) > 0, and 
therefore every u G & a {si) satisfies |<(si,u)| < tt/2. Hence, the surface area of ^„(si) 
is always larger than half the surface area of Si and therefore 

' (133) 



v < 2 



□ 



Combining (|133p with (|132j) and (|13ip proves the lemma. 

E.5.1 Proof of Lemma IE. 41 

We begin with the following decomposition 

Pr[£ x ] = Pr[£ x n £ s ] + Pr[£ x H £§] 

< Pr[£ s ] + Pr[£ Xl n £%] + Pr[£ Xa n £§] + Pr [£ (Xl ,x 2 ) n £ Xa n £ Xj n E%] 

< Pr[£ s ] + Pr[£ Xl ] + Prfot 2 ] + Pr [£ (Xl ,x 2 ) n £ x , n % n £§] . 

The proof of Lemma lE.41 now follows by showing that for every 5 > and 0.3 > e > 
there exists an n' 2 (5, e) > such that for all n > n' 2 (5, e) 



Pr[£ Xl ]<5, *€{1,2} 

Pr[f (Xl)X2) n^nf Xl nf X2 ] <3<5. 



(134) 
(135) 
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Proof of (I134p : We give the proof for £ Xl . Due to the symmetry the proof for £x 2 
then follows by similar arguments. Let £ Xl (j) be the event that Ui(j') does not have 
a typical angle to Si, i.e. 



£ Xl (j) = {(si,s 2 ,Ci,C 2 ) : |cos<(ui(j),Si) - y/l-2~^\ > e\A-2- 2 *i} 



Then, 



Pr[f Xl ] = Pr[f Xl |Si = Si] 

2 nR i 

= Pr P| £ Xl (j) Si = si 

= Pr[f Xl 0')|Si=si] 

^ H Pr[f Xl (i)] 

^(Pr^ Xl (l)]) 2 ^ 
= (1-Pr[^(l)]) 



(136) 



where in a) we have used that the probability of £ Xl (j) does not depend on Si = si, 
and in b) we have used that all Ui(j') have the same distribution. To upper bound 
(|136p we now rewrite f Xi (l) as 



£ Xl (l) = {(si,s 2 ,Ci,C 2 



cos ■ 



<(ui(l),si) - \/l-2- 2fl i| < e\/l- 2- 2fl i } 

{(si,s 2 ,Ci,C 2 ) : Vl " 2- 2R i(l - e) < cos <(ui(l), si) < y/l - 2-^(1 + e)} 
{(si,s 2 ,Ci,C 2 ) : cos0i, max < cos<(ui(l),si) < cos6>i jmin } , 



where we have used the notation 



cos0i, max ^ Vl-2- 2 «i(l-e) and cos liJnin 4 ^1 - 2" 2 ^ (1 + e). 
Hence, since Ui(l) is generated independently of Si and distributed uniformly on Si, 

Cn(01,max) Cn(01,min) 



Pr[£ Xl (l)] 
Combining (|137p with (|136p gives 



(137) 



1 



C n (9l max ,min J 



a) 

< I exp 



C n (n) 

l,max) Cra(01,min/ 

cv^) 



2" K i 



62 




2 n Rl r (i + 1 ) ( gm^^max L _ 1 ^2 ^ \ _ gm^gl^ 

nr (^±i) v^r y cos6*i >max \ n ' m / cos6>i jmin 

/ r (- + l) / 2 71 ( i? l+ 1 °S2( sin6, l,ma X )) / 1 o 

\ nr (^±±) //r \ sin 6>i imax cos 6>i imax \ » 



2" +log 2 (sin 6»i , min ) ) 



SmCl,min COS 01 



where in a) we have used that 1 — x < exp(— x), and in b) we have lower bounded 
C n {6\ jUiax ) / C n {ir) and upper bounded C n {0\ jm \ n ) / C n {i:) according to Lemma fE. 171 It 
now follows from sphere-packing and -covering, see e.g. [25J, that for every e > we 
have Pr[£xJ — » as n — > do. More precisely, this holds since the exponent on the RHS 
of (I138D grows exponentially in n. This follows since on the one hand for large n 

r(f + i) i 



and on the other hand the term 

2n(-Rl+log 2 (sin01 ima x)) / 1 \ 2 n (- R l+ 1 °g2( sin6 'l,min)) 

1 tan 2 6»i : 



sm0 limax cos0i )max V n J sm 0i,mi n cos 0i, m in 

grows exponentially in n. The latter holds since first of all 

1 tan 2 8 1 m ax ) ~ 1 for large n, 

n ' J 

second, the denominators of the fractions are independent of n, and third since 

Ri + log 2 (sin6>i jmax ) >Ri + log 2 (sin6>i >min ) , 
with i?i + log 2 (sin 0i jinax ) > 0. That R\ + log 2 (sin 9i >max ) > can be seen as follows. 



Iog 2 (sin0 limax ) = -log 2 (yl - cos 2 6»i jmax ) 

a) ilog 2 (2- 2 ^+ £ (2- e )(l-2- 2 ^)) 
1 



<-~log 2 (2- 2fll ) 
= R\, 

where in a) we have used the definition of cos#i imax . □ 
Proof of fj 135[) : By the notation in (|128j) we have 

K,u*) 



cos<(u*,u^) 



l U lllll U 2ll 

(v x v 2 (si,s 2 ) + v\ (si, v 2 ) + v 2 (vi,s 2 ) + (vi,v 2 )) , (139) 



(138) 
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where we recall that V\ is a function of ||si|| and cos<(si,u^) and similarly v<i is a 
function of ||s2|| and cos <(s 2 , u?i). Now, define the four events 



A x 
A 2 
A 3 
Aa 



(si,s 2 ,Ci,C 2 ) 
(si,s 2 ,Ci,C 2 ) 
(si,s 2 ,Ci,C 2 ) 
(si,s 2 ,Ci,C 2 ) 



(Sl,8 2 ) 



> 4e 



u u 



V-2 



u u 



(si,v 2 ) 

(S2,V X ) 



(vi,v 2 ) 



> e 



> e 



> e 



i u iiniu 2 

Note that by (fT39l) . £(Xi,x 2 ) C (-4i U .4 2 U .A3 U Aa). Thus, 

Pr [5 (Xl ,x 2 ) n s% n £ Xl n £ X2 ] < Pr[Ai n eg n £ Xl n £ X2 ] + Pr [A 2 n f§ n £ Xl n £ Xa ] 

+ Pr [A 3 n £ § n £ Xl n £ X J + Pr [A n £§ n £ Xl n £ X J 

< Pr[Ai|£ s nf Xl n£ x J + Pr[.A 2 |£§] + Pr[^ 3 |^] 
+ Pt[A a \S$\. (140) 

The four terms on the RHS of (I140p are now bounded in the following two lemmas. 
Lemma E.20. For e < 0.3 

PrfAl^n^n^iJ =o. 

Proof. We first note that the term in the definition of Ai can be rewritten as 

Z/li/ 2 



u u 



— (si, s 2 ) = cos<(si,u*) cos <(s 2 , u 2 ) cos <(si, s 2 ). 



(141) 



We can now upper and lower bound the RHS of (|14ip for (si , s 2 , C\, C 2 ) G £g n£ Xi n£ X2 
by noticing that (si, s 2 ,Ci,C 2 ) G £g implies 

|cos<(si,s 2 ) — p\ < pe, 

that (si, s 2 , Ci, C 2 ) G £ Xl implies 

^1-2-2^1 -cos<(si,uJ 

and that (si, s 2 , C\, C 2 ) G £ Xa implies 



Hence, combined with (I14ip this gives 
^lf 2 



e) 3 < 



< eA/l-2- 2 ^, 
^1-2-2% -cos<(s 2 ,u^) < e\fl - 2- 2J fe. 

(si,s 2 ) <p(i + e) 3 , (si,B2,Ci,c 2 ) g ££nf Xl n£ X2 . 



The LHS can be lower bounded by p(l — 3e) < p(l — e) 3 , and the RHS can be upper 
bounded by p(l + e) 3 < p{l + 4e) whenever e < 0.3. Hence, for e < 0.3 



uT u 



< Ape < 4e. 



□ 
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Lemma E.21. For every 5 > and e > there exists an n'^(5,e) such that for all 
n > n' A (5, e) 



Pr[A 2 \£i] < 5, 



Pr[A 3 \£%] < S, 



Pr[-4 4 |££] < S. 



Proof. We start derivation of the bound on A 2 . To this end, we first upper bound the 
inner product between si and V2. Let s^p denote the projection of si onto the subspace 
of W 1 that is orthogonal to S2, and that thus contains V2. Hence, 



UT 



(si,v 2 ) 



COS <(Si,Ui) 
b) 

< |cos <(si, u t ) 
Si v 2 



Si 
l S l| 



Si 
l S l| 



V2 
\ U 2 \ 
V2 
ll v 2| 



< 



< 



ll S l|| 
Sl,P 



l v 2| 
V 2 



|si|| ||v 2 
Si P v 2 



S 1,P|| ||V2| 

cos<(si p, v 2 ) 



(142) 



where a) follows by the definition of V\ and b) follows since by the definition of v 2 we 
have 1 1 V2 1 1 < || u 2ll- By (|142p it now follows that 

Pi[A 2 \£g] < Pr[(Si, S 2i V u V 2 ) : |cos <(Si, P , V 2 )| > e|£T|] 



< Pr 



(Si, S2, ^L, ^2 



7T 



--<(Si, P ,v 2 ; 



> e 



cc 

Go 



-Si,S2 



Pr 



7T 



<(si,p,v 2 ; 



> e 



(Si,S 2 ) = (si,s 2 ),^ 



where in the last line we have denoted by Pr^ (•••!•••) the conditional probability 
of the codebooks ^1 and *?f 2 being such that \n/2 — <(si 5 p,V 2 )| > e given (Si,S 2 ) = 
(si,s 2 ) and (si,s 2 ) G £§. To conclude our bound we now notice that conditioned on 
(Si, S 2 ) = (si, s 2 ), the random vector V 2 /||V 2 || is distributed uniformly on the surface 
of the centered R n_1 -sphere of unit radius that lies in the subspace that is orthogonal 
to s 2 . Hence, 



Pr[A 2 \£ c s ] < E Sl)S 



2C„_i(vr/2-e) 



< 



< 



< 



C„_i(vr) 

2C ra _i(7r/2-e) 
C„_i(vr) 

2T(^) sin("- 2 )(7r/2-e) 
(n - l)r (§) v^F cos(^/2 - e) 

2P(^) 
(n-l)r(f) v^Fcos^/2-e)' 
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Upper bounding the ratio of Gamma functions by the asymptotic series of Lemma 
IE.181 gives for every e > that Prf^l^s] ~* as n ~* °°- By similar arguments it 
also follows that Pr[y43|£g] — > as n — ► oo. 

To conclude the proof of Lemma [E.2H we derive the bound on A4. The derivations 
are similar to those for A.2- First, define by v^p the projection of vi onto the subspace 
of W 1 that is orthogonal to S2. As in (|142p we can show that 



1 



UT U 



TIT ( v i' v 2! 



< |cos<(vi p, v 2 ) 



(143) 



from which it then follows, using | cosx| < \n/2 — x\, that 



Pr[A4\£g\ < E Sl ,s 3 ,«i 



7T 



<(vi, P ,v 2 ; 



> e 



(Si,S 2 



The desired bound now follows from noticing that conditioned on (Si,S 2 ) = (si,s 2 ) 
and ^fj = C\i the random vector V 2 /||V 2 || is distributed uniformly on the surface of 
the centered M n_1 -sphere of unit radius that lies in the subspace that is orthogonal to 
s 2 . Hence, similarly as in the derivation for A2 



Pr[A±\£ c s ] < E Sl>Sa ,«i 



2C n _i(vr/2-e) 



C ft _i(7r) 



< 



2r(B ±i; 



l)r (§) ^cos(vr/2 



Upper bounding the ratio of Gamma functions by the asymptotic series of Lemma IE, 181 
gives for every e > that Pr[Ai|£g] — > as n — > 00. □ 

Combining Lemma IE.20I and Lemma IE.21I with H140j) gives that for every 5 > and 
0.3 > e > there exists an n'^(S, e) such that for all n > n'^(8, e) 

Pr[£ (Xl ,x 2) n^n£ Xl n£: X2 ] < 35. □ 

E.5.2 Proof of Lemma lE7rl 

The proof follows from upper bounding Pr [t/|£ x J as a function of R%. First, note that 

Pr[0|%] = Pr[S|U*/0] 



Pr[g| a (Si,«i) = 1], 



(144) 



where the second equality holds because the conditional distribution of the codewords 
conditional on u* / is invariant with respect to permutations of the indexing of the 
codewords. The desired upper bound is now obtained by decomposing Q into sub-events 
e {2, 3,..., 2 nR *}, where 



Qj = {(si,s 2 ,Ci,C 2 ,z) : cos <(w,ui (,?')) > A} 
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By (|144|) we now have 



2 nR i 






a(Si,ifi) = l 


J=2 





2 nK l 

< Pr[g i | a (Si,«' 1 ) = l] 

i=2 

<2^ Pr[g2| a (Si,«i) = l] 

Rl Cn(arccos A) 
CJtt) ' 



(145) 



where in the third step we have used that Pr[C/j|q(sj,Cj) = 1] is the same for all j G 
{2, 3, . . . 2 nRl } because the conditional distribution of u\(j) given Q(si,d) = 1 does 
not depend on j G {2, 3, ... , 2 ni?1 } and where in the last step we have upper bounded 
the density of Ui(2), conditional on fl(Si,^fi) = 1, by Lemma fE.191 Thus, combining 
(|145p with Lemma |E. 171 gives 



Pr[£|££j < 2 nRl • 2 



r(f + i) (i - a 2 )^- 1 )/ 2 



2r(§ + i) 



nr (2±i) V^AVl - A 2 



2 n( J R. 1 +l/21og 2 (l-A 2 )) 



Replacing the ratio of the Gamma functions by the asymptotic series of Lemma IE. 181 
establishes (fTTT]) . □ 

E.5.3 Proof of Lemma HOI 

The proof follows by upper bounding Pr^fi^ n as a function of R\ + i? 2 . To 
this end, define 

<f(si,s 2 ,Ci,C 2 ) = (ft(si,Ci),? 2 (s2,C 2 )) • 

By a symmetry argument, which is similar to the one in the proof of Lemma lE.71 we 
obtain 

Pr[g\S^ n ££J = Pr[0|?(Si, S 2 , ^, <*f 2 ) = (1, 1)] . (146) 
The desired upper bound is now obtained by decomposing Q into subevents Gj,£, where 



Gj,e = {(si,s 2 ,Ci,C 2 ,z) : cos<(ui(j),u 2 (£)) > 6, cos <(y, «iui(j) + a 2 u 2 (f)) > A}, 
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for j G {2, 3, . . . 2 nRl } and £ G {2,3, . . . 2 ni?2 }. Hence, by (HMD 



Pr^l^n^J = Pr 



2 nR i 2 nH 2 



u u s» 



f(S 1 ,S 2 ,^i, < r 2 ) = (l,l) 

i=2 £=2 
2 nfl i 2 nil 2 

^ E E Prfe 1 ^(Si,S 2 ,^ 1 ,^ 2 ) = (1,1)] 

i=2 1=2 

< 2^ 1+R2 )Pr[g 2i2 |^(S 1 ,S2,^i,^ 2 ) = (1,1)] , (147) 

where a) follows since conditioned on <f(Si, S 2 , ^l, ^2) = (1, 1), the laws of <(Ui(j), U 2 (£)) 
and <(Y, aiV 1 (j)+a 2 V 2 (£)) do not depend on j G {2, 3, . . . , 2 nRl } or £ G {2, 3, ... , 2 ni?2 }. 
We now rewrite the probability Pr[£? 2i2 |<f(Si, S2, ^1, ^2) = (1, 1)] : 

Pr[g 2j2 |e(Si,S 2 ,^i,^ 2 ) = (l,l)] 

= Pr[(Si,S 2 ,Ui(l),U 2 (l),Ui(2),U 2 (2),Z) are such that G 2a occurs 

|?(Si,S2,tfl,tf 2 ) = (1,1)] 



(s 1 ,s2)eK n xR™ /((Si,S 2 ,Ui(l),U 2 (l),Z) = (si,s 2 ,ui,u 2 ,z) 

(ui,u 2 )G5i xS 2 

zeR " |?(Sa,S 2 ,^i,^ 2 ) = (l,l)) 

Pr[cos<(Ui(2),U 2 (2)) > 6, cos <(y, aiUi(2) + a 2 U 2 (2)) > A 

|e(Si,S 2 ,^i,^ 2 ) = (l,l),(Si,S 2 ,Ui(l),U 2 (l),Z) = (si,S2,ui,u 2 ,z)] 
d(si,s 2 ,ui,u 2 ,z) 



(si,s 2 )e 

(ui,u 2 )G5ix5 2 



/((Si,S 2 ,Ui(1),U 2 (1)) = ( Sl ,s 2 ,ui,u 2 ) 



|?(Si,s 2 ,<*?i,<r 2 ) = (1,1)) 

Pr[cos<(Ui(2),U 2 (2)) > 6, cos <(y, aiUi(2) + a 2 U 2 (2)) > A 

|?(Sa,S 2 ,^i,^ 2 ) = (l,l),(Si,S 2 ,U 1 (l),U 2 (l)) = ( Sl ,s 2 , Ul ,u 2 )] 
d(si,s 2 ,ui,u 2 ), 

where in the last step we have used that the probability term does not depend on z. 
To upper bound the integral we now upper bound this probability term. 



Pr 



cos<(Ui(2),U 2 (2)) > 6,cos<(y,aiU 1 (2) +a 2 U 2 (2)) > A 

?(S 1 ,S 2 ,^ 1 ,^ 2 ) = (l,l),(S 1 ,S 2 ,U 1 (l),U 2 (l)) = (s 1 ,s 2 ,u 1 ,u 2 ; 

jiu^es^sr. / AlxA2 ((U!(2),U 2 (2)) = (iii, u 2 )|?(Si, S 2 , <# h <T 2 ) = (1,1), 

cos<(y)a' | , a 1 +° 2 U2)>A (Si , S 2 , U X (1) , U 2 (1) ) = (si, S 2 , U U U 2 )) d(ui, U 2 ) 
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L,u 2 )e Sl xSr. / Al (Ui(2) = Ui|?(S ll S 2 , < ifi ) tf 2 ) = (1,1), 

''COS <(ui,U 2 )>0, 

co S <(y,a 1 ui+a 2 u 2 )>A (S 1 ,S 2 ,Ui(l),U 2 (l)) = (si, S 2 , Ui, U 2 )) 

•/ A2 (U 2 (2) =u 2 |?(Si,S 2 ,%,^2) = (1,1), 

(S 1 ,S 2 ,U 1 (1),U 2 (1)) = (si,s 2 ,ui,u 2 ))d(ui,u 2 ) 

Lwesixs,-. f Xl (Ui(2) = uxIftCSi,^,) = l,(Si,Ui(l)) = (si.ua)) 



'cos <(ui,U 2 )>0, 

cos <(y,aiUi+a 2 u 2 )>A 



/ A2 (U 2 (2) = u 2 | ?2 (S 2 ,<rf 2 ,) = 1,(S 2 ,U 2 (1)) = (s 2 ,u 2 )) d(ua,u 2 ) 



a ) f 2 2 

< /(uLUaJe^xSa: 7 x ' 7 x d(ui, u 2 ) 

• / cos<(ui,u 2 )>e, r l L/ nl 7 U r 2 L/ nV 7 U 

cos <(y,«iui+a 2 u 2 )>A 



6) C n (arccos9) C n (arccos A) 

C n (7T) ' C n (7T) ' U48j 

where in a) we have used Lemma [E.19l and in b) we have used that under distributions of 
Ui(2) and U 2 (2) that are independent of y and uniform over S± and 5 2 respectively, the 
angles <(Ui(j), U 2 (^)) and <(y, aiUi(j)+a 2 U 2 (£)) are independent. Thus, combining 
(jUSD with ([T37j) gives 



Pr-Tcl^ n; c l^o«(flx+ifa) , C ra (axccos9) C ra (arccosA) 

Pr^ Xl n£ X2 j<2 -4— ^ ctf^j - ' 

And combining (|149p with Lemma IE. 171 gives 

n (Rl+ « 2) . r(f + i) (i - e^-w r(| + i) (i-a^-d/^ 



(149) 



nr (2±i) v^e nr (a±i) v^fa 



/ r + i ) \ ' 

4- 



r (t + 1 ) 1 1 9 n( fll+ R 2 + | log 2 ((l-e 2 )(l-A 2 ))) 

nr(^±i)^Fi e v / r^e2A v / r^A2 



Replacing the ratios of the Gamma- functions by their asymptotic series in Lemma [E. 181 
finally establishes (|117p , □ 



F Proof of Theorem 14.5 



The high-SNR asymptotics for the multiple-access problem without feedback can be 
obtained from the necessary condition for the achievability of a distortion pair (Di, -D 2 ) 
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in Theorem 14.11 and from the sufficient conditions for the achievability of a distortion 
pair (D\,D 2 ) deriving from the vector-quantizer scheme in Theorem 14.41 

By Theorem 14.41 it follows that any distortion pair (1)1,1)2) satisfying D\ < a 2 , 
D 2 < a 2 and 

£>! > a 2 ^- (150) 
D 2 > a 2 — (151) 
DjiW 2 Ar ( 1 ^ 2 ) , (152) 

where 



is achievable. If 



-^X 1 -^ 1 ' <153) 

N N 

lim =- = and lim =- = 0, (154) 

N^oP^i N->oP 2 D 2 

then for N sufficiently small, (|150p and (|151 1) are satisfied. Consequently, for A" suffi- 
ciently small any pair satisfying (|152p and (|154p is achievable. We next show that if 
the pair {D\,D 2 ) satisfies (j!52|) and (|154p . then p —* p as A" —* 0. To show this, we 
note that if (Di,D 2 ) satisfies (|152p then 

a N A N 

D 2 < cj 4 — and D x < a 4 ^. (155) 

" PiDi ~ P 2 D 2 V ' 

Combining (|155p with (I153h gives that if in addition to (|152p the pair (Di,D 2 ) also 
satisfies (fl"55|) . then p ^ p as N ^ 0. Thus, if (D 1 ,D 2 ) satisfies (IT52D and (HMD, then 

jr™ iV 2 ( X ~ ^ )• ( 156 ) 

Now, let (DJ(cr 2 ,p,Pi,P2,A r ),^>2(°' 2 'P' P i' P 2,A r )) be a distortion pair resulting 
from an optimal scheme and let (D^D^) be the shorthand notation for this distortion 
pair. By Theorem 14.11 we have that 



R Sl MDl,D* 2 ) < ilog 2 (l + Pl + P2 + 2P ^ ) • (157) 
If (DJ,D|) satisfies 

AT AT 
lim — — = and lim — — = 0, (158) 

N^O PiDl N^O P 2 Dl 

then for A" sufficiently small 

R Sl ,s 2 (Dt,D* 2 ) = ilog+ ( ^~f 2) ) , (159) 

by Theorem [37TI and because (D*,!)!) € ^2- From (|157p and (|159p we thus get that if 
(Dl,D*) satisfies IjlBBJl . then 

lim Pl+P2 + 2pVT ^ D* 1 D* 2 > o\l - p 2 ). (160) 
Combining ()156[) with (|160p yields Theorem 14.51 □ 
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G Proof of Theorem 14.61 



Our analysis of the expected distortion for the superimposed scheme is based on a 
genie-aided argument, similar as in the analysis of the vector-quantizer scheme. This 
argument is described more precisely now. 



G.l Genie- Aided Scheme 

In our genie- aided argument, the genie assists the decoder. An illustration of this de- 
coder is given in Figure [T3l In addition to the channel output Y that is observed orig- 



U? 



us 



Y 






«J1 




5* 

Y 




3» 




u 2 


> 



Figure 13: Genie-aided decoder. 



inally, the decoder is now also provided with the transmitted codeword pair (U^USJ. 
Based on (U|,USJ and Y, the decoder then estimates the source pair (81,82) and 
thereby ignores the guess (Ui,U2) produced in the first step of the original decoder. 
The estimate of this genie-aided decoder is denoted by (S^ , S!f ) and is given by 



711 U; + 712^ + 713Y 
721 U2 + 722U1 + 723 Y, 



(161) 
(162) 



where the coefficients 7^ are as defined in (|33p . We now show that under certain 
rate constraints, the normalized asymptotic distortion of this genie-aided scheme is the 
same as for the originally proposed scheme. The key argument is stated in the following 
proposition. 

Proposition G.l. For every 5 > and < e < 0.3 there exists an n'(5,e) > such 
that for all n > n'(5, e), 



n 



|Si 



1 

< — E 

n 



Sl-Sf|| 2 + 



(163) 



whenever (R\,R,2) is in the rate region 7Z'(e) given by 



TZ'(e) 



i f p[ 2 \\vi\\ 2 a 



P 2 ) + N' 



R 2<\ log 2 



/52 2 I|U 2 || 2 (1 



- p 2 ) + N' 



N'(l 



P 2 ) 



K\e 



K 2 e 
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p ^ o . 1 , ^f 2 ||Ui|| 2 + ^||Ui|| 2 + 2^||Ui||||U 2 || + ^ \ \ 

where in (1163[) ^ and £ 2 depend only on a 2 , 713, Pi, P 2 iV, and where in the 
expression of1Z'(e) the terms k±, k 2 and K3 depend only on Pi, P 2; N', and p, and 
where N' and (3[, (3' 2 are as given in flMJ) , ([35]) anc? ([36]) respectively. 

Proof. See Section EU □ 

From Proposition IG.ll it now follows easily that the expected distortion asymp- 
totically achievable by the genie-aided scheme is the same as the expected distortion 
achievable by the original scheme. 

Corollary G.l. //(Pi,i? 2 ) satisfy 

ft<W ;2||Uil|2(1 -^ )+iV ' 



2 n>(i-p 2 ) 



2 toi V - p 2 ) 

1 ^f||U 1 || 2 + / af||U 1 || 2 + 2p#$||U 1 ||||U 2 || + N> 



2 b2 V N'{1 - p 2 ) 



then 



lim — E 

n— >oo n 



IS1-S1I 



< lim — E 

n— >oo 7i 



I Si - 



Gi|2 



Proof. Follows from Proposition IG.ll by first letting n — > 00 and then e — > and 
(5^0. □ 

It follows by Corollary IG.ll that to analyze the distortion achievable by our scheme 
it suffices to analyze the genie-aided scheme. This is done in Section IG.3I 

G.2 Proof of Proposition IG.ll 

The proof of Proposition lG~T1 consists of upper bounding the difference between ^E [||Si — S 
and ~E[||Si — S^|| 2 ]. Since the two estimates Si and Sp differ only if (Ui,U 2 ) 7^ 
(U*, U 2 ), the main step is to upper bound the probability of a decoding error. This is 
what we do now. 

Let the error event be as defined in (|89h - (|91h for the vector-quantizer scheme. 
The probability of £fj is upper bounded in the following lemma. 

Lemma G.l. For every 5 > and < e < 0.3, there exists an n'^S^e) 6 N such that 
for all n > e) 

Pr[%] < 115 whenever (Ri,R 2 ) G 1l\e). 



ill 2 ! 
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Proof. The proof follows from restating the decoding problem for the superimposed 
scheme in the form of the decoding problem for the vector-quantizer scheme. That is, 
we seek to rewrite the channel output in the form 



Y = #UJ + (3' 2 XJ* 2 + Z', 



(164) 



with an additive noise sequence Z' that satisfies the properties needed for the analysis 
of the vector-quantizer scheme. This representation is obtained by first rewriting the 
source sequences as 



51 = (l-aip)U^ + aiU£ + Wi 

5 2 = (1 - a 2 p)XJ* + a 2 \]\ + W 2 , 



(165) 
(166) 



where a\ is defined in (|37p . a 2 is defined in (|38p . and p is defined in (|26p . Combining 
(|165p and (|166l) with the expressions for Xi and X 2 in (j29H and with Y = Xi + X 2 + Z 
yields the desired form of (|164p with 

(3[ = (ai(l - a\p) + Pi + a 2 a 2 ) 
2 = (a 2 (l - a 2 p) + (3 2 + aiai) , 



and with 



Z' = aiWi + a 2 W 2 + Z. 



For the additive noise sequence Z' it can now be verified that for every 5 > and e > 
there exists an n'(8, e) > 0, such that for N' as in (j34|) and for all n > n'(<5, e) we have 
that 



Pr 



1 



7? 



r'l|2 



iV' 



< AT'e 



and that 



Pr 



(U*,Z') | < nJa 2 (l - 2-™i)N'e 



> 1 - 5, (167) 

>l-(5, iG{l,2}. (168) 
for sufficiently large 



Condition (|168|) follows since for a\ and a 2 , given in ([57]) and 
n, we have with high probability that 

(U*,W,)«0 Vi,i€{l,2}. 

Conditions (|167|) and (|168|) are precisely those needed in the proof of the achievable 
rates for the vector-quantizer scheme. Hence, the upper bound on the probability of a 
decoding error in the vector-quantizer scheme given in Lemma IE. II can be adopted to 
the superimposed scheme. This yields Lemma lG.ll □ 

To ease the upper bounding of the difference between ^E[||Si — Si || 2 ] and ^E[||Si — 
S^"|| 2 ] we now state three more lemmas which upper bound different norms and inner 
products involving Si, Si and Sf. The first lemma gives an upper bound on the 
squared norm of Si — S*p. 
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Lemma G.2. Let the reconstructions Si and S'p be as defined in (|31h and (I161|) . 
TTien, mt/t probability one 



|Si - Sf || 2 < 16na 2 . 



Proof. 



|Si - S? || 2 



*M|2 



= ||7il(Ui-U^)+ 7l2 (U 2 -U^ 

= 7i 2 il|Ui - UJII 2 + 2 7ll7l2 (Ui - UJ,U 2 
+ 7l 2 2 ||U 2 -U^|| 2 

< 7 n ||Ui - U^|| 2 +2 7ll7l2 ||Ui - U|||||U 2 - Ual 



<4ncr 2 

+ 7l 2 2 ||U 2 -U*|| 2 

<4n<7 2 

< 4n<r 2 ( 7 u + 7i2 ) 2 

< IQna 2 , 

where in the last step we have used that < 711,712 < !• 



<4n<r 2 



□ 



For the next two lemmas, we reuse the two error events £s and £z which were 
defined in ([92]) and ([93]) for the proof of the vector-quantizer scheme. We then have: 



Lemma G.3. For every e > 
1 



-E 

n 



Si, Sf - Si ) < a 2 (e + 17 Pr [5s] + (17 + e) Pr [£$] ) . 



Proof. 



Ie 

n 



Si, Sx — Si 



-E 

n 

+ Ie 

n 



Si,Sf -Si) SsUSfj Pr[£ s u5 6 



Si, S-l — Si 



Pr 



a) 1 

< -E 

n 



|Siir + 1 Si 



G 



u 



Pr^sU^] 



iE[||Si|| 2 |£: s ] Pr[£ s ] + ~E[||Si|| 2 |£ s n%] Pr[^n^] 

Ur "Sf -Sill 2 SsUsJ Pr^sU^] 



+ -E 

n 



b) 



+ 16cT 2 (Pr[ < S s ]+ Pr[5e]) 
< a 2 (e + 17Pr[f s ] + (17 + e) Pr [S t ] ) , 



(169) 
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where in the first equality, the second expectation term equals zero because by we 
have 1 1 — si|| =0 and by the norm ||si|| is bounded. In a) we have used (I12ip . 
and in b) we have used Lemma IE.101 Lemma IG.2I and the fact that conditioned on S g 
we have ||si || 2 < n<r 2 (l + e). □ 

Lemma G.4. For every e > 



Ie 

n 



Proof. 



|Si| 



*?ll 2 



< 4(V(1 + 4 7 i 3 ) + 9713 (Pi + P 2 + JV)(1 + e)J Pr[f c ] 
+ 36 7 i3 f (Pi + P2) Pr[£ s ] + NPv[£ z ] + (Pi + P 2 + iV)e 



n 



IS! || 2 



|Sf|| 2 



n 

+ i E 

n 



|Si| 



|Sf|| 2 



<Ie 

n 



|Si 



Si I 

1 1 2 



I Si 



u 

Gi|2 



S. 



u 



Pr 



5. 



u 



G||2 



u 



(170) 



where the last inequality follows since conditional on we have Si = Sf and therefore 

Si|| 2 — HS'pll 2 = 0. To upper bound the RHS of (|17U|) . we now upper bound the 

"G 
1 



difference ||Si|| 2 



IS?" 2 - 



|Sif - ||Sf || 2 = 7iil|Ui|| 2 + 2711712 (Ui,U 2 ) + 2711713 (£1, Y 

+ 7i 2 2 ||U 2 || 2 + 2712713 (U 2 , Y) + 7i 2 3 l|Y|| 2 

- 7nl|U 1 || 2 - 2711712 (UJ,U5) - 2711713 (VI Y) 

- 7i 2 2 ||U^|| 2 - 2712713 (US, Y) - 7 2 3 ||Y|| 2 

= 2711712 (Ui,U 2 ) +2711713 (Ui, Y^> + 2712713 (U 2 , Y 

<ri<r 2 

- 2711712 (U]^ -2711713 (UJ, Y) - 2712713 {UI Y) 

>— ncr 2 



< 47n7i 2 n(j + 2711713 <^Ui - Ui,Y 
+ 2712713 (tT 2 -U^,Y 

a) 

< 4711712nd 2 + 2 7 ii7i 3 ( ||Ui -Ui|| 2 +||Y|| 2 ) 

<Ana 2 

+ 27i27i3( ||U2-U2l| 2 +l|Y|| 2 ) 

<4ncr 2 

< 47n7i 2 no- 2 + 2713(711 +7i 2 )(4no- 2 + ||Y|| 2 ) 

< 4no- 2 + 47i 3 (4na 2 + ||Y|| 2 ), 



(171) 
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where in o) we have used (I121D . and in the last inequality we have used that < 
7li)7i2 < 1- We now upper bound the squared norm of Y on the RHS of (|171|) in 
terms of Si, S2, U*, U2 and Z: 

||Y|| 2 < a\ ||Si|| 2 + 2 (aiSi.ftUD + 2 ( ai Si, a 2 S 2 ) + 2 (aiSi, f3 2 U* 2 ) 
+ 2 (aiSi, Z) + /5i 2 ||Ut|| 2 + 2 (/3iUj,a 2 S 2 > + 2 (ftUj./fcUS) 
+ 2 (/3iUj, Z) + a|||S 2 || 2 + 2 (a 2 S 2 ,/3 2 U^) + 2 (a 2 S 2 , Z) 
+ /? 2 2 ||U^|| 2 + 2(/3 2 U 2 \Z) + ||Z|| 2 

< 9 (a 2 ||Si|| 2 + ai||S 2 || 2 +/3 2 ||U1|| 2 +/3 2 2 ||U^|| 2 + ||Z|| 2 ) 

< 9 (a 2 ||Si|| 2 + ai||S 2 || 2 + nP 1 + nP 2 + ||Z|| 2 ) . (172) 

where a) follows from upper bounding all inner products by (|121j) . Thus, combining 
CE72]) with (IT7I]) gives 

||Si|| 2 - [|Sp || 2 < 4 tic 2 + 16 7 i 3 nCT 2 + 36n 7 i 3 (Pi + P 2 ) 

+36 7l3 (a 2 ||Si|| 2 + a 2 ||S 2 || 2 + ||Z|| 2 ) . (173) 

And combining (| 173p with (|170p gives 
1 



-E 

n 



|Si|| 2 - ||S? || 2 



<4a 2 Pv[£ t ] + 16713a 2 Pr[£ c ] 
+ 36713 (Pi +P 2 )Pr[£: ( j] 

+ 36 7 i 3 (aTiE[||Si|| 2 |^] Prfo] 

+ a 2 iE[||S 2 || 2 |^] Prfo] 



+ lE[||Z|| 2 |£: (j ]Pr[^]). (174) 



It now remains to upper bound the expectations on Si, S 2 and Z on the RHS of (|174|) . 
Since Si, S 2 and Z are each Gaussian, their corresponding terms can be bounded in 
similar ways. We show here the derivation for Si. 

iE[||Si|| 2 |^] Pr[£ t} ]=h[\\S 1 f\S tj nS s ] Pv[£ t n£ s ] 



+ U[\\S 1 \\ 2 \£ i] n£i] Pr[£ e n££] 



< — E [||S! || 2 |^s] Pr[£ s ] 
+ a 2 (l + e)Pr[£: t j] 

<a 2 (e+ Vv[£ s ]) + a 2 {l + e)Vx[£ t ] , (175) 

where in the last step we have used Lemma |E. 101 For the expectations on S 2 and Z, 
we similarly obtain 

IeOIS,!! 2 ^] Prfo] <a\e+ Pr[£ s ]) + a\l + e) Pr [£ t ] , (176) 
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and 



-E[||Z|| 2 |£ 6 ] Pr[S t ]<N(e+PT[S z ]) + N(l + €)Pr[S t ]. 
Thus, combining (fT75|) - (fTTTj) with (fT7lj) gives 

^e[||Si|| 2 - ||Sf|| 2 ] <4<T 2 Pr[£ c ] + 16713a 2 Prf^] 

+ 36 7 i3 (Pi + P 2 )Pr[£ tj ] 
+ 36 7 i3 ((Pi + P 2 )(e + Pr[5 s ]) 

+ (Pl + P 2 + iV)(l + e)Pr[%] 

+ iV(e+ Pr[f z ]) N 



(177) 



< 4(<r 2 (l + 4 7 i 3 ) + 9713 (Pi + P 2 + JV)(1 + e)) Pr[£a] 

+ 36 7l3 f (Pi + P 2 ) Pr[5s] + AT Pr[f z] + (Pi + P 2 + iV)e ) . □ 



Based on Lemma IG.3I and Lemma IG.41 the proof of Proposition IG.ll now follows 
easily. 



Proof of Proposition IG.il 



Ie 

n 



IS1-S1I 



Ie 

n 



I Si - 



G||2 



n 



|Si 



|Si 



Sf|| 2 



- E ||Si| 
n 



2E 



Si, S] 



+ E 



|Si| 



2 Ie 

n 



EniSifl +2E 



Si, Si 



Si, s l 

+ Ie 

n 



G 11 2 



ISil 



|Sf|| 2 



a) 



< 2a 2 (e + 17Pr[f s ] + (17 + e) Pr [%] ) 

+ 4(a 2 (l + 4 7 i 3 ) + 9 7l3 (Pi + P 2 + N)(l + e)) Pr[£ e ] 

+ 36 7 i3 ((Pi + P 2 ) Pr[f S ] + iVPr[£ z ] + (Pi + P 2 + ] 

= £1 Prfc,] + 6 Pr[«Ss] + £3 Pr[5z] + fa, (178) 

where in step a) we have used Lemma lG.3l and Lemma lG.41 and where £ £ {1, 2, 3, 4}, 
depend only on <r 2 , 713, Pi, P 2 and iV. Combining (I178p with Lemma lE.ll Lemma IE. 21 
and Lemma lE.31 gives that for every 5 > and 0.3 > e > 0, there exists an n'(5, e) > 
such that for all (Pi, R2) £ 7£' (e) and n > n'(5, e) 



|Si 



■n 



Si-S^l 2 <£<f + &2'e, 



where ^ and £' 2 depend only on a 2 , 713, Pi, P 2 and N. 



□ 
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G.3 Upper Bound on Expected Distortion 

We now derive an upper bound on the achievable distortion for the proposed vector- 
quantizer scheme. By Corollary IG.ll it suffices to analyze the genie-aided scheme. 
Using that Sf = 7 nU* + 712U2 + 713Y, we have 



n 



Sf|| 2 ] =i(E[||S 1 || 2 ] -2 711 E[(S 1 ,UJ)]-2 712 E[(S 1 ,U5)] 

-2 7l3 E[(Si,Y)]+ 7 f 1 E[||Ut|| 2 ] + 2 7ll7l2 E[(U?,U 2 <)] 
+ 2 7ll7 i3E[(Ut,Y)]+ 7l 2 2 E[||U^|| 2 ] 

+ 2 7l27l 3E[(U^Y)]+ 7l 2 3 E[||Y|| 2 ]). (179) 

Some of the expectation terms are bounded straightforwardly. In particular, we have 
E [|| Si || 2 ] = na 2 , E[||U^|| 2 ] = na 2 (l - 2~ 2R ^), and E[||U^|| 2 ] = na 2 {\ - 2~ 2R ^. For 
three further terms we take over the bounds from the analysis of the vector-quantizer 
scheme. That is, by Lemma |E. 141 we have that for every 5 > and < e < 0.3 and 
every positive integer n 

-E[(S 1 ,Vl)}>a 2 (l-2- 2 ^)-( 1 (S,e) 
n 

= cu-Ci(<5,e), (180) 

where e) is such that lim^^o Ci(^j e ) = 0- By Lemma |E. 151 we have that for every 
5 > and < e < 0.3 there exists an n' 2 (S, e) 6 N such that for all n > n' 2 (5, e) 

-E[(\Jl,V* 2 )} < a 2 p(l - 2- 2iJl )(l - 2- 2if2 ) +( 2 (5,e) 

= ki 2 + C 2 (5,e), (181) 



n 



where £2(8, e) is such that lim,5 e _>o C2(<5, e) = 0. And by Lemma lE.161 we have that for 
every 5 > and < e < 0.3 there exists an n'(S, e) £ N such that for all n > n'(8, e) 

-E[(S 1 ,V* 2 )]>a 2 p(l-2- 2 ^)-Ca(S,e) 
n 

= c 12 -US,e), (182) 

where £3(5, e) is such that lim^^o C3 (<5, e) = 0. Next, recalling that Y = a\Si + (3i\J\ + 
a 2 S 2 + A2U2 + Z, gives 

-E[(Si,Y>] = -faiE[||Si|| 2 ] +/3iE[(Si,UJ>] + a 2 E[(Si,S 2 )] 

+ ^E[(Si,U5)] + E[(Si,Z>] 

=0 

a) 

> aid 2 + Pi (en - (1(6, e)) + a 2 p<J 2 + (3 2 (c 12 - (s(S, e)) 

= c 13 -U(5,e), (183) 

where in a) we have used (|180p . (|18ip and (|182p . and where £4(5, e) is such that 
lim,5 e _»o C4(<^ e ) = 0- For the remaining terms in (|179p . it can be shown, similarly 
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as for (1180P and (|182l) . that for every 5 > and < e < 0.3 there exists an n"(5, e) G N 
such that for all n > n"(£, e) 



-E[<Si, 
n 


Ui>] 


< cu+C5(^,e) 


(184) 


~E[(S 2: 

n 


U?>] 


< c 2 i +C 6 (^,e) 


(185) 


-E[(S lt 

n 


U5>] 


< ci 2 +C 7 (5,e) 


(186) 


^E[(S 2; 


U5>] 


< c 22 +Cs(<5,e), 


(187) 



n 

where e), j G {5, . . . 8}, are such that lim^o e) = 0. Using (fl8T|) and ([TBI]) 

- (j!87j) . we now get that for every 5 > and < e < 0.3 there exists an h\{5, e) G N 
such that for all n > ni(<5, e) 

iE[(Ut,Y)] = i(a 1 E[(Ut,S 1 )]+/3 1 E[||Ul|| 2 ]+a 2 E[(Ut,S2)] 

+ &E[(U^U£)] + E[(Uj,Z)] 

=o 

< qi (kn + Cb(<5, e)) + /3i kn + a 2 (c 2 i + Ce(5, e)) 

= ki 3 + Ci(<5,e), (188) 

where Ci(<^ e ) is such that lim^^o Ci(<5> e ) = 0- Similarly, it can be shown that for every 
5 > and < e < 0.3 there exists an h 2 (S, e) G N such that for all n > h 2 (S, e) 

-E[(U5,Y)] < k 23 + C 2 (<5,e), (189) 
n 

where C, 2 {5,e) is such that lim^g^o C2 {8, e) = 0. And finally, we have that for every 
5 > and < e < 0.3 there exists an h^(6, e) G N such that for all n > n 3 (5, e) 

— E [|| Y|| 2 ] =ifa?E[||S 1 || 2 ] +2a 1 /3 1 E[(S 1 ,Ut)] + 2a 1 a 2 E[(S 1 ,S 2 )] + 2a 1 /3 2 E[(S 1 ,U;)] 

+ 2ax E[(S l5 Z)] +/3 2 E [||U* || 2 ] + 2/? l£ * 2 E[(UJ, S 2 >] + 2(5 1 f3 2 £[{\5\, XJ* 2 )] 
=0 

+ 2A E[(U|, Z)] +a 2 E [||S 2 || 2 ] + 2a 2 /3 2 E[(S 2 , U^)] + 2a 2 E[(S 2 , Z)] 



=0 

+ /3 2 E[||U^|| 2 ] +2/3 2 E[(lg : Z)]+E[||Z|| 2 ] 

=0 

< a\a 2 + 2ai/3i(cn + Cs(M) + 2aia 2 /wx 2 + 2ai/3 2 (ci 2 + C7^,e)) 
+ /3 2 k n + 2/3 ia2 (c 21 + Ce(5, e)) + 2/? 1 /? 2 (k 12 + C 2 (<5, e)) 
+ a^ 2 + 2a 2 /3 2 (c 22 + C 8 (<5, e)) 
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+ /3 2 k 22 + iV 

= k 33 + C 3 (5,e)), (190) 

where C3(<5> e ) is such that lims e _> £3(5, e) = 0. Thus, combining (|180p - fj 1 83 1) and 
(TL881) - (fT90l) with (fT79l) gives that for every 5 > and < e < 0.3 there exists an 
n'(8, e) G N such that for all n > n/(<5, e) 



Ie 

n 



|Si - Sf || 2 



< a 2 - 2711C11 - 2712C12 - 2713C13 + 7nkn + 2711712 k i2 
+ 27ii7i 3 ki 3 + 7 2 2 k 2 2 + 7i27i3k23 + 7i3 k 33 + C'(^ e) 

= cr - 2711C11 - 2712C12 - 2713C13 

' 7ii \ 

+ (711 712 713) K I 712 \+C{8,e) 



713 



= a 2 - 2711C11 - 2712C12 - 2713C13 



en 

+ (711 712 713 ) KK _1 I C12 I +?(5,e) 



C13 



a - 2711C11 - 2712C12 - 2713C13 



Cll 

(711 712 713) ( C12 ) +('(5,e) 

Cl3 

= a 2 - 711C11 - 712C12 - 713C13 + C'(*> e )> ( 191 ) 

where we have used the shorthand notation K for K(i?i, R2), and where in a) we 
have used the definition of the coefficients 7^' in (|33p . and where £'(<5, e) is such that 
lim^^o C'(^> e ) = 0- Now, letting in (|19ip first n — > 00 and then 5, e — > 0, and combining 
the result with Corollary IG.ll gives 



lim E 

n— >oo 



IS1-S1H 2 



< a 2 - 711C11 - 712C12 - 713^13, 



whenever (i?i,i?2) satisfy 



2 ^ V ^'(1 " P) 



2 &2 V - p 2 ) 

o ^ R 1, ^/3i 2 ||Ui|| 2 + /32 2 ||U2|| 2 + 2^/? 2 ||Ui||||U2||+iV' 

/?i + i?2 < - log 2 
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